Esci dai Frame

	Deep Learning (a.a. 2023/24) Secondo Semestre
Elenco dei File nel deposito
Sommario Contenuti 1.Novita' 2.Programma del Corso 3.Testi di Riferimento 4.Link Utili 5.Diapositive delle lezioni 6.Progetti ed Esercizi Proposti
Novita' INtroTestInItinere1_aa2022_23_woAnswers.pdf Slides for Question Examples and Topics for the MidTerm of the academic year 2023-24. A Python notebook describing the application of CNNs to the MNIST dataset has been published in the Lab and Exercises Section of this page. Please download and study the exercise: it is part of the Exam evaluation. ANNOUNCEMENT: First MidTerm Test The First MidTerm Test of the Deep Learning Course for the academic year 2023-24 will be held on: Monday April 29th, 2024, at 14:00-17:00 Preparatory activities (examples of closed answer questions as well as open questions, exercises, examples) will be presented and carried out during the next lessons preceding the MidTerm date. During these lessons the part of the Course Program onto which the test will insists (i.e. the topics that are evaluated in the first MidTerm test) will be presented, documented and discussed. Students are invited to register on Delphi. ANNOUNCEMENT: On Monday March 25 2024, the lesson will be held only remotely on the MS Teams of the Course, from 12:00 to 13:00, as for an institutional duty of the responsible. For the students that will not be able to joint, the lesson will be recorded, and it will be made avaliable in the Course MS Teams channel. Lessons will continue regularly from Wednsady March 27th (h. 9:30). ANNOUNCEMENT The lesson foreseen at 14:00 on Monday, March 18th is CANCELLED, as for personal problems of the course responsible. Lessons will continue regularly on Wednesday March, 20th. ANNUNCIO: Il Corso avra' inizio regolarmente a partire dal 6 Marzo 2023, secondo il seguente orario: LUNEDI', h. 14:00-16:00 (Aula 19 Edifici Aule Macroarea di Scienze) MERCOLEDI', h. 9:30-11:30 (Aula 19 Edifici Aule Nuove della Macroarea di Ingegneria) GIOVEDI'', h. 11:30-13:30 (Aula 19 Edifici Aule Macroarea di Scienze) The teams of the course under MS Teams is: "23_24_BASILI-WEB_MIN_AND_RETR_Deep_Learning". Please check your membership on MS Teams. The course will start on March 6 according to the following: Course TIMETABLE: MONDAY, h. 14:00-16:00 (Aula 19 Classrooms Science Macroarea) WEDNESDAY, h. 9:30-11:30 (Aula 19 Classrooms Engineering Macroarea) THURSDAY, h. 11:30-13:30 (Aula 19 Classrooms Science Macroarea) Please register for the Course on Delphi. Le diapositive delle lezioni saranno pubblicate durante il ciclo delle lezioni su queste pagine. Il Corso insiste sulle ricerche ed i progetti innovativi del Semantics Analytics Group (SAG), che si occupa di Deep Learning e Natural Language Processing nella progettazione ed ingegnerizzazione di Sistemi Software Avanzati di Intelligenza Artificiale. Tali sistemi sono applicati a processi predittivi nella interpretazione e ricerca di documenti, nella sicurezza in rete, nella analisi dei Social Network e nei processi di Digital Transformation nell'Industria e nella Medicina. Sono attive alcune sperimentazioni e progetti presso il SAG Laboratory for Semantics Analytics, da cui sono emanate annualmente alcune Borse di Studio e Premi di Laurea. Sara' possibile discutere in dettaglio le diverse Tesi con il coordinatore di SAG, prof. Roberto BASILI, o con il responsabile tecnico del Laboratorio, prof. Danilo CROCE. L'orario di ricevimento, diverso da quello dei Corsi, e' da concordare con i docenti via e-mail.
Programma Segue il programma preliminare del Corso che sara' messo a punto ed finalizzato al termine delle lezioni del Corso. Section I: Advanced Machine Learning and Deep Learning. Deep Learning and Artificial Intelligence. Supervised methods. Statistical Learning Theory: PAC learnability. Kernel-based Learning. Task dependent kernels. Deep Learning with Neural Networks: Perceptron, Multilayer Percetrons, Deep Neural Networks. Image Processing with Deep Learning Architectures. Section II: Neural Language Processing. Neural Models for Language Processing. Language Models and Recurrent Networks. Attention-based Neural Networks. Transformers: BERT. Introduction to the main software platforms for the development of DL software: PyTorch. Large Language Models. Prompting and Instruction Tuning. Neural models for Textual Inference: from few shot learning to prompting. Section III: Deep Learning Applications. Visual Object Recognition and Automatic Image Captioning. Information Extraction. Fake News Detection. Opinion Mining and Sentiment Analysis. Brand Reputation Analysis and Marketing.
Testi di Riferimento ML/DL - Introduction to Deep Neural Learning, Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, MIT Press, 2016. Neural Language Processing. Foundation Models for Natural Language Processing, Gerhard Paaß and Sven Giesselbach, Springer Nature, URL of the book. Probability and Computing. Introduction to Probability for Computing, Mor Harchol-Balter, Cambridge University Press, URL of the book, 2024. ML. Pattern Recognition and Machine learning, C. Bishop. Springer. 2006. Web IR - Bing Liu, Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. 2nd Edition, July 2011, Springer. Dispense fornite dal docente
Lezioni (Lessons Slides) Lesson 0: Deep Learning - a.a. 2023-24: Introduction: Course Organization and Exam Modalities. Short history of Large Language Models: perspectives for business processes. Lesson 1: Introduction to Web Mining & Retrieval. Some slides of Lesson 1 refer to the discussion of the link: "A visual introduction to ML", slide 24). Lesson 1.1: Machine Learning: target problems and major paradigms. Lesson 2: Machine Learning Metrics and Evaluation (part I: metrics for Text Classification). Lesson 3: Language Modeling - an Introduction to Hidden Markov Models for Sequence Labeling. Complementary Materials (Non mandatory): Lesson 3a. Parameter Estimation for Language Modeling: the Baum-Welch algorithm. Lesson 3b. Parameter Estimation and Rare Phenomena in Language Modeling. Lesson 4: (A gentle) Introduction to PAC learning and VC dimension. The slides used for the Course have been postedited from a kindly published version by Ethem Alpaydin, that you can find HERE. C. Burges's Tutorial on SVM and VC dimension. D. Haussler discussion of PAC Learning, 1999. Valiant L. G. A Theory of the Learnable, Communications of the ACM, Volume 27 Issue 11, Nov. 1984 Pages 1134-1142 . Lesson 5 and 6: Support Vector Machines and Kernels (Full package). An animated Perceptron. Dan Klein's tutorial on Lagrange methods for the SVM optimization problem. Section II - Introduction to Neural Networks and Deep Learning Architectures Lesson 7 An Introduction to Neural Learning. The MultiLayer Perceptron: defining and training MLPs. Lab 1 - Introduction to Keras: the XOR example. Lab 2a - A Linear classifier and a MLP for image classification over the MNIST dataset in Keras. Lab 2b - A Linear classifier and a MLP for image classification over the MNIST dataset in Pytorch. Lesson 8 Deep Learning: Complex Architectures and Tasks: Convolutional Neural Networks References and resources: Gradient Descent and Perceptron training, "A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm", Quoc V. Le (Google), (fino a sezione 6 compresa) Backpropagation on NNs, "Sparse autoencoder", from the lessons of Andrew Ng (Stanford University, USA), (pages 1-12) Advanced Architectures, "A Tutorial on Deep Learning, Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks", by Quoc V. Le (Google Brain) Deep Learning, "Learning Deep Architectures for AI", Yoshua Bengio Dept. IRO, Universite' de Montreal, CA. An interesting in-depth analysis on CNN Convolutional Neural Networks for Visual Recognition: CNNs for Visual Recognition on GitHub Software packages for Deep Learning: TensorFlow Anaconda Torch Keras Tensorflow Pytorch Lesson 9: Recurrent Neural Networks, Encoder and Decoder Networks
Link Utili NLP with Spacy.io Google TensorFlow A Scalable and Distributed Python Framework for Deep Neural Network Design and Development Hugging Face: a repository of large scale Deep Neural Network Models and Datasets KeLP: a Java framework for Kernel-based Structured Learning SciKit Learn
Laboratory Material and Exercises Introduction to NLP with Spacy: Informatiopn Extraction from Texts: morphological information, normalized tokens (Lemmas), grammatical dependencies. Slides of the March 14th lesson Python Notebook with exercises: running Spacy and questions. Data set for Tests: Question Classification corpus. Python Book for the exercise on HMM for POS tagging. Development of the HMM model by probability estimation over a corpus. Development of the tagged corpus with Spacy. Kernel-based Learning in KeLP . Python Book for the exercise on HMM for POS tagging. Development of the HMM model by probability estimation over a corpus. Development of the tagged corpus with Spacy. Application of CNNs to the MNIST dataset: Introduction to CNNs, overview and computation of Convolutional and Pooling layers with an application of CNNs on MNIST dataset.