Esci dai Frame

	Deep Learning (a.a. 2023/24) Secondo Semestre
Elenco dei File nel deposito
Sommario Contenuti 1.Novita' 2.Programma del Corso 3.Testi di Riferimento 4.Link Utili 5.Diapositive delle lezioni 6.Progetti ed Esercizi Proposti
Novita' ANNOUNCEMENT: Dates of Exams of the September session - The Autumn Session Exams of the Deep Learning Course for the academic year 2023-24 will be held on: September 5th, Third Final Test, h. 10:00 September, 25th, Fourth Final Test, h. 10:00 The classes will be requested and will be announced in the course Home Page asap. The test will be carried out in the standard two stage manner: (1) Closed Answer questions (2) Open Question Students are invited to register on Delphi. Results of the Second Final Test (July 15th, 2024). In order accept the evaluation, please use the Exam thread in the MS Teams of the Course to confirm acceptance. In this case the registration in Delphi will be carried out without further meetings. Students that want to improve evaluation through an oral discussion or a project should contact the coordinator of the Course via email to schedule a meeting. Results of the Second MidTerm test and of the First Final Test. . In order accept the evaluation, please use the Exam thread in the MS Teams of the Course to confirm acceptance. In this case the registration in Delphi will be carried out without further meetings. Students that want to improve evaluation through an oral discussion or a project should contact the coordinator of the Course via email to schedule a meeting. ANNOUNCEMENT: Exam Summer Sessions The Summer Session Exams of the Deep Learning Course for the academic year 2023-24 will be held on: June 18th, Second MidTerm and First Final Test, h. 15:00 July, 15th, Second Final Test, h. 10:00 The classes will be requested and will be announced in the course Home Page asap. The test will be carried out in the standard two stage manner: (1) Closed Answer questions (2) Open Question Students are invited to register on Delphi. Results of the First MidTerm test : the admitted student can register to the Second Mid Term.. Introduction to the Second MidTerm test : Overview of the 2023-24 Program, Closed Answer Questions and Open Question Topics.. BERT Lab material: Application of BERT model on Question Classification dataset: Overview of BERT model family and intro to Huggingface repository. Fine-Tuning a Linear Layer atop a Pre-trained BERT Model on the QC dataset, using the following dataset: Question Classification dataset (both train and test set) The data used in the discussion of the Orange framework can be found HERE. Please download and study the exercise: it is part of the Exam evaluation. Data and Python notebook describing the application of LSTMs to the BIO tagging and Entity Extraction have been made available in the Lab and Exercises Section of this page. Please download and study the exercise: it is part of the Exam evaluation. Slides for Question Examples and Topics for the MidTerm of the academic year 2023-24. A Python notebook describing the application of CNNs to the MNIST dataset has been published in the Lab and Exercises Section of this page. Please download and study the exercise: it is part of the Exam evaluation. ANNOUNCEMENT: First MidTerm Test The First MidTerm Test of the Deep Learning Course for the academic year 2023-24 will be held on: Monday April 29th, 2024, at 14:00-17:00 Preparatory activities (examples of closed answer questions as well as open questions, exercises, examples) will be presented and carried out during the next lessons preceding the MidTerm date. During these lessons the part of the Course Program onto which the test will insists (i.e. the topics that are evaluated in the first MidTerm test) will be presented, documented and discussed. Students are invited to register on Delphi. ANNOUNCEMENT: On Monday March 25 2024, the lesson will be held only remotely on the MS Teams of the Course, from 12:00 to 13:00, as for an institutional duty of the responsible. For the students that will not be able to joint, the lesson will be recorded, and it will be made avaliable in the Course MS Teams channel. Lessons will continue regularly from Wednsady March 27th (h. 9:30). ANNOUNCEMENT The lesson foreseen at 14:00 on Monday, March 18th is CANCELLED, as for personal problems of the course responsible. Lessons will continue regularly on Wednesday March, 20th. ANNUNCIO: Il Corso avra' inizio regolarmente a partire dal 6 Marzo 2023, secondo il seguente orario: LUNEDI', h. 14:00-16:00 (Aula 19 Edifici Aule Macroarea di Scienze) MERCOLEDI', h. 9:30-11:30 (Aula 19 Edifici Aule Nuove della Macroarea di Ingegneria) GIOVEDI'', h. 11:30-13:30 (Aula 19 Edifici Aule Macroarea di Scienze) The teams of the course under MS Teams is: "23_24_BASILI-WEB_MIN_AND_RETR_Deep_Learning". Please check your membership on MS Teams. The course will start on March 6 according to the following: Course TIMETABLE: MONDAY, h. 14:00-16:00 (Aula 19 Classrooms Science Macroarea) WEDNESDAY, h. 9:30-11:30 (Aula 19 Classrooms Engineering Macroarea) THURSDAY, h. 11:30-13:30 (Aula 19 Classrooms Science Macroarea) Please register for the Course on Delphi. Le diapositive delle lezioni saranno pubblicate durante il ciclo delle lezioni su queste pagine. Il Corso insiste sulle ricerche ed i progetti innovativi del Semantics Analytics Group (SAG), che si occupa di Deep Learning e Natural Language Processing nella progettazione ed ingegnerizzazione di Sistemi Software Avanzati di Intelligenza Artificiale. Tali sistemi sono applicati a processi predittivi nella interpretazione e ricerca di documenti, nella sicurezza in rete, nella analisi dei Social Network e nei processi di Digital Transformation nell'Industria e nella Medicina. Sono attive alcune sperimentazioni e progetti presso il SAG Laboratory for Semantics Analytics, da cui sono emanate annualmente alcune Borse di Studio e Premi di Laurea. Sara' possibile discutere in dettaglio le diverse Tesi con il coordinatore di SAG, prof. Roberto BASILI, o con il responsabile tecnico del Laboratorio, prof. Danilo CROCE. L'orario di ricevimento, diverso da quello dei Corsi, e' da concordare con i docenti via e-mail.
Programma Segue il programma preliminare del Corso che sara' messo a punto ed finalizzato al termine delle lezioni del Corso. Section I: Advanced Machine Learning and Deep Learning. Deep Learning and Artificial Intelligence. Supervised methods. Statistical Learning Theory: PAC learnability. Kernel-based Learning. Task dependent kernels. Deep Learning with Neural Networks: Perceptron, Multilayer Percetrons, Deep Neural Networks. Image Processing with Deep Learning Architectures. Section II: Neural Language Processing. Neural Models for Language Processing. Language Models and Recurrent Networks. Attention-based Neural Networks. Transformers: BERT. Introduction to the main software platforms for the development of DL software: PyTorch. Large Language Models. Prompting and Instruction Tuning. Neural models for Textual Inference: from few shot learning to prompting. Section III: Deep Learning Applications. Visual Object Recognition and Automatic Image Captioning. Information Extraction. Fake News Detection. Opinion Mining and Sentiment Analysis. Brand Reputation Analysis and Marketing.
Testi di Riferimento ML/DL - Introduction to Deep Neural Learning, Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, MIT Press, 2016. Neural Language Processing. Foundation Models for Natural Language Processing, Gerhard Paaß and Sven Giesselbach, Springer Nature, URL of the book. Probability and Computing. Introduction to Probability for Computing, Mor Harchol-Balter, Cambridge University Press, URL of the book, 2024. ML. Pattern Recognition and Machine learning, C. Bishop. Springer. 2006. Web IR - Bing Liu, Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. 2nd Edition, July 2011, Springer. Dispense fornite dal docente
Lezioni (Lessons Slides) Lesson 0: Deep Learning - a.a. 2023-24: Introduction: Course Organization and Exam Modalities. Short history of Large Language Models: perspectives for business processes. Lesson 1: Introduction to Web Mining & Retrieval. Some slides of Lesson 1 refer to the discussion of the link: "A visual introduction to ML", slide 24). Lesson 1.1: Machine Learning: target problems and major paradigms. Lesson 2: Machine Learning Metrics and Evaluation (part I: metrics for Text Classification). Lesson 3: Language Modeling - an Introduction to Hidden Markov Models for Sequence Labeling. Complementary Materials (Non mandatory): Lesson 3a. Parameter Estimation for Language Modeling: the Baum-Welch algorithm. Lesson 3b. Parameter Estimation and Rare Phenomena in Language Modeling. Lesson 4: (A gentle) Introduction to PAC learning and VC dimension. The slides used for the Course have been postedited from a kindly published version by Ethem Alpaydin, that you can find HERE. C. Burges's Tutorial on SVM and VC dimension. D. Haussler discussion of PAC Learning, 1999. Valiant L. G. A Theory of the Learnable, Communications of the ACM, Volume 27 Issue 11, Nov. 1984 Pages 1134-1142 . Lesson 5 and 6: Support Vector Machines and Kernels (Full package). An animated Perceptron. Dan Klein's tutorial on Lagrange methods for the SVM optimization problem. Section II - Introduction to Neural Networks and Deep Learning Architectures Lesson 7 An Introduction to Neural Learning. The MultiLayer Perceptron: defining and training MLPs. Lab 1 - Introduction to Keras: the XOR example. Lab 2a - A Linear classifier and a MLP for image classification over the MNIST dataset in Keras. Lab 2b - A Linear classifier and a MLP for image classification over the MNIST dataset in Pytorch. Lesson 8 Deep Learning: Complex Architectures and Tasks: Convolutional Neural Networks References and resources: Gradient Descent and Perceptron training, "A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm", Quoc V. Le (Google), (fino a sezione 6 compresa) Backpropagation on NNs, "Sparse autoencoder", from the lessons of Andrew Ng (Stanford University, USA), (pages 1-12) Advanced Architectures, "A Tutorial on Deep Learning, Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks", by Quoc V. Le (Google Brain) Deep Learning, "Learning Deep Architectures for AI", Yoshua Bengio Dept. IRO, Universite' de Montreal, CA. An interesting in-depth analysis on CNN Convolutional Neural Networks for Visual Recognition: CNNs for Visual Recognition on GitHub Software packages for Deep Learning: TensorFlow Anaconda Torch Keras Tensorflow Pytorch Lesson 9: Recurrent Neural Networks, Encoder and Decoder Networks Lesson 10: Word embeddings as distributional semantic lexical representations. Lesson 11 Neural Word Embeddings: between Language Modeling and Lexical Acquisition Lesson 12 Attention: the introduction to Transformers Lesson 13 Deep Learning: NLP tasks, Benchmarking Datasets and Evaluation Lesson 14 Beyond Transformers: Decoder only Architectures. From zero-shot and few-shot learning to large language model prompting. Lesson 15 Beyond Transformers: Instruction Learning. From zero-shot to Instruction Learning. Lesson 16 LLMs Trends: LoRA and RAG. Lesson 17 Short Introduction to Opinion Mining. See also: Tutorial on Opinion Mining (by B. Liu) Book Chapter on Opinion Mining and Sentiment Analysis (by Bo Pang and Lillian Lee) Introduction to the Second MidTerm test : Overview of the 2023-24 Program, Closed Answer Questions and Open Question Topics..
Link Utili NLP with Spacy.io Google TensorFlow A Scalable and Distributed Python Framework for Deep Neural Network Design and Development Hugging Face: a repository of large scale Deep Neural Network Models and Datasets KeLP: a Java framework for Kernel-based Structured Learning SciKit Learn
Laboratory Material and Exercises LAB 01: Introduction to NLP with Spacy: Information Extraction from Texts: morphological information, normalized tokens (Lemmas), grammatical dependencies. Slides of the March 14th lesson Python Notebook with exercises: running Spacy and questions. Data set for Tests: Question Classification corpus. LAB 02: POS tagging through HMM. Python Book for the exercise on HMM for POS tagging. Development of the HMM model by probability estimation over a corpus. Development of the tagged corpus with Spacy. LAB 03: Kernel-based NLP: Question Classification. Java KeLP libraries for Question Classification via SVMs. Jave classes, Question Classifier and "How to" for Java Installation. LAB 04: Application of CNNs to the MNIST dataset: Introduction to CNNs, overview and computation of Convolutional and Pooling layers with an application of CNNs on MNIST dataset. LAB 05: BIO tagging with LSTMs: The task of BIO tagging and Entity Extraction, Training a Long-Short-Term-Memory Network in Keras with Exercises LAB 06: Application of BERT model on Question Classification dataset: Overview of BERT model family and intro to Huggingface repository. Fine-Tuning a Linear Layer atop a Pre-trained BERT Model on the QC dataset, using the following dataset: LAB 07: Question Classification dataset (both train and test set) LAB 08: The LangChain Framework for Sequence Tasks: Overview and Usage of the LangChain framework for sequences. Focus on the core modules: Models, Prompts and Chains. LAB 09: The Application of Llama 3 model for relation extraction in the medical domain: Python Notebook with the overview of the Relation Extraction task and management of the Huggingface repository. Exploit the Llama 3 capabilities investigating 0-Shot, One-Shot and Few-Shot paradigms, using the Clinkart dataset. LAB 10: The ExtremITA : Bringing Language Models to the Extreme: the EvalIta 2023 system.