Esci dai Frame

	Deep Learning (a.a. 2024/25, ex Web Mining and Retrieval) Semester Second
List of Files on the Page
Content Summary 1.News 2.Course Program 3.Reference Textbooks 4.Useful Links 5.Slides of the lessons 6.Projects and Exercises
Announcement: The First MidTerm Test of the course is scheduled on April 28th, 2025, and it will be held in Classroom C10, at 13:00. The test will focus on all the topics discussed during the lessons up to April 17th, 2025. Students are invited to register to the exam at the Delphi pages. News Announcement: The lesson of Thursday April, 3rd will focus on a Lab exercise in the use of SVM and Kernel Learning software on different classification Tasks. Proposed Exercises on HMM modeling. NEW Course Lesson Timetable: The Course lessons will continue regularly since March 24, 2025, according to the following agenda that subsitute the previous one: MONDAY, h. 14:00-16:00 (Room C10 Classroom Building in the Engineering Macroarea) WEDNESDAY, h. 9:30-11:30 (Room B16 Classroom Building in the Engineering Macroarea) THURSDAY, h. 11:30-13:30 (Room B2 Classroom Building in the Engineering Macroarea) As no change has been possible, the lesson of Thursday, March 20, 2025 will be held as usually in class C12 at 9:30. Announcement OLD time table: The Course lessons will start regularly since March 3, 2025, according to the following timetable: MONDAY, h. 14:00-16:00 (Room C10 Classroom Building in the Engineering Macroarea) WEDNESDAY, h. 9:30-11:30 (Room B16 Classroom Building in the Engineering Macroarea) THURSDAY, h. 9:30-11:30 (Room C12 Classroom Building in the Engineering Macroarea) The teams of the course under MS Teams is: "BASILI-8067802-DEEP_LEARNING". Please check your membership on MS Teams. The course will start on March 3rd according to the following: OLD Course TIMETABLE: MONDAY, h. 14:00-16:00 (Room C10 Classrooms Science Macroarea) WEDNESDAY, h. 9:30-11:30 (Room B16 Classrooms Engineering Macroarea) THURSDAY, h. 9:30-11:30 (Room C12 Classrooms Science Macroarea) Please register for the Course on Delphi. Teaching materials, such as papers PDFs or slides will be published on this page during the Course activities. The Course refers to research and projects of the Semantics Analytics Group (SAG) that makes research about Deep Learning and Natural Language Processing in the design and engineering of Generative AI systems and complex AI Services. These systems provide predictive and intelligent decision making functionalities document search and interpretation, in the Knowledge Integration and Summarization in the Intellligent Cybersecurity, in the Social Network Analysis and in all processes of Digital Transformation in the industry or in specific domains, such as medicine or banking/fintech. Active experimentations and projects are detailed at SAG Laboratory for Semantics Analytics, that regularly financially supports Applied Research Internships or Laurea Thesis awards.
Course Contents This is the preliminary Course syllabus updated across the Course and finalized and the end of the lessons. Section I: Advanced Machine Learning and Deep Learning. Deep Learning and Artificial Intelligence. Supervised methods. Statistical Learning Theory: PAC learnability. Kernel-based Learning. Task dependent kernels. Deep Learning with Neural Networks: Perceptron, Multilayer Percetrons, Deep Neural Networks. Image Processing with Deep Learning Architectures. Section II: Neural Language Processing. Neural Models for Language Processing. Language Models and Recurrent Networks. Attention-based Neural Networks. Transformers: BERT. Introduction to the main software platforms for the development of DL software: PyTorch. Large Language Models. Prompting and Instruction Tuning. Neural models for Textual Inference: from few shot learning to prompting. Section III: Deep Learning Applications. Specialized Large Language Models in the Medicine, Biology, Nuclear Energy or Banking domains. Visual Object Recognition, Automatic Image Captioning, Visual Question Answering. Information Extraction. Fake News Detection. Opinion Mining and Sentiment Analysis. Brand Reputation Analysis and Marketing.
Reference Text Books ML/DL - Introduction to Deep Neural Learning, Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, MIT Press, 2016. Neural Language Processing. Foundation Models for Natural Language Processing, Gerhard Paaß and Sven Giesselbach, Springer Nature, URL of the book. Probability and Computing. Introduction to Probability for Computing, Mor Harchol-Balter, Cambridge University Press, URL of the book, 2024. ML. Pattern Recognition and Machine learning, C. Bishop. Springer. 2006. Web IR - Bing Liu, Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. 2nd Edition, July 2011, Springer. Further Teaching Materials provided by the Teacher.
Lessons Slides In this section the slides of the different lessons and other teaching materials will be published. Lesson 0: Deep Learning - a.a. 2024-25: Introduction: Course Organization and Exam Modalities. Lesson 1: Introduction to Deep Learning in the perspective of Web Data Mining. Some slides of Lesson 1 refer to the discussion of the link: "A visual introduction to ML", slide 24). Lesson 1.1: Machine Learning: target problems and major paradigms. Short history of Large Language Models: perspectives for business processes. Lesson 2: Machine Learning Metrics and Evaluation (part I: metrics for Text Classification). Complementary Materials (Non mandatory): Short introduction to the Vector Space Model. Basic Notion of Linear Algebra Entropy and Machine Learning. Basic Notion of probabilistic norms for dissimilarity estimation. Lesson 3 : Probability and Learning: an introduction to Naive Bayes classifiers. Lesson 4: Language Modeling - an Introduction to Hidden Markov Models for Sequence Labeling. Complementary Materials (Non mandatory): Lesson 4a. Parameter Estimation for Language Modeling: the Baum-Welch algorithm. Lesson 5: (A gentle) Introduction to PAC learning and VC dimension. The slides used for the Course have been postedited from a kindly published version by Ethem Alpaydin, that you can find HERE. M. Kearns and Vazirani, An introduction to Computational Learning Theory, MIT Press (Chapt. 1, 2) C. Burges's Tutorial on SVM and VC dimension. D. Haussler discussion of PAC Learning, 1999. Valiant L. G. A Theory of the Learnable, Communications of the ACM, Volume 27 Issue 11, Nov. 1984 Pages 1134-1142 . Lesson 6: Support Vector Machines. An animated Perceptron. Dan Klein's tutorial on Lagrange methods for the SVM optimization problem. Lesson 7: Kernel Methods. Lesson 8: Tree Kernels and NL Inference Tasks. SVM, Kernel Lab: an introduction to KeLP Files for the Lab Exercise. It includes the datasets, the sw library and useful examples. Section II - Introduction to Neural Networks and Deep Learning Architectures Lesson 9: An Introduction to Neural Learning. The MultiLayer Perceptron: defining and training MLPs. READINGS Nature Paper by Hinton and colleagues: Rumelhart, D. E., Hinton, G. E., and Williams, R. J., Learning representations by back-propagating errors. 1986 Rumelhart, D. E., Hinton, G. E., and McClelland, J. L. A general framework for Parallel Distributed Processing, In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 45-76. 1986 Hinton, G. E., McClelland, J. L., and Rumelhart, D. E., Distributed representations., In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 77-109. 1986 Rumelhart, D. E., Hinton, G. E., and Williams, R. J., Learning internal representations by error propagation. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 318-362. Exercises Lab 1 - Introduction to Keras: the XOR example. Lab 2a - A Linear classifier and a MLP for image classification over the MNIST dataset in Keras. Lab 2b - A Linear classifier and a MLP for image classification over the MNIST dataset in Pytorch. Lesson 10 Deep Learning: Complex Architectures and Tasks: Convolutional Neural Networks Laboratory Work: Application of CNNs to the MNIST dataset Introduction to CNNs, overview and computation of Convolutional and Pooling layers with an application of CNNs on MNIST dataset. OLD exercises (Pre 2024-25) Lab 3b - A CNN for image classification over the MNIST dataset in Pytorch. Lab 3a - A CNN for image classification over the MNIST dataset in Keras. References and resources: Gradient Descent and Perceptron training, "A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm", Quoc V. Le (Google), (fino a sezione 6 compresa) Backpropagation on NNs, "Sparse autoencoder", from the lessons of Andrew Ng (Stanford University, USA), (pages 1-12) Advanced Architectures, "A Tutorial on Deep Learning, Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks", by Quoc V. Le (Google Brain) Deep Learning, "Learning Deep Architectures for AI", Yoshua Bengio Dept. IRO, Universite' de Montreal, CA. An interesting in-depth analysis on CNN Convolutional Neural Networks for Visual Recognition: CNNs for Visual Recognition on GitHub Software packages for Deep Learning: TensorFlow Anaconda Torch Keras Tensorflow Pytorch Lesson 11: Recurrent Neural Networks: architectures and applications BIO tagging with LSTMs: The task of BIO tagging and Entity Extraction, Training a Long-Short-Term-Memory Network in Keras on the Annotated Corpus for Named Entity Recognition using GMB (Groningen Meaning Bank) dataset, using the following file: NER Dataset Lesson 12: Word embeddings as distributional semantic lexical representations. Lesson 13 Neural Word Embeddings: between Language Modeling and Lexical Acquisition Lesson 14 Attention: the introduction to Transformers Sentiment Analysis with BERT on IMDB dataset: Training and evaluating a BERT-based model on movie reviews. Exercises include training set size variation, MLP architecture edits, fine-grained analysis and evaluation Lesson 15 Deep Learning: NLP tasks, Benchmarking Datasets and Evaluation Lesson 16 Beyond Transformers: Decoder only Architectures. From zero-shot and few-shot learning to large language model prompting. Lesson 17 Beyond Transformers: Instruction Learning. From zero-shot to Instruction Learning.
Link Utili NLP with Spacy.io Google TensorFlow A Scalable and Distributed Python Framework for Deep Neural Network Design and Development Hugging Face: a repository of large scale Deep Neural Network Models and Datasets KeLP: a Java framework for Kernel-based Structured Learning SciKit Learn
Laboratory Material and Exercises In this section Exercises and Solutions for the typical exam questions and tests will be published.