|
Deep Learning (a.a. 2024/25, ex Web Mining and Retrieval) Semester Second |
|
|
List of Files on the Page |
|
Content Summary
1.News
2.Course Program
3.Reference Textbooks
4.Useful Links
5.Slides of the lessons 
6.Projects and Exercises
|
Announcement: The final test of the Course for the Summer session, that is Second MidTerm Test, First Final Test and the Secondo Final Test, are scheduled according to the following agenda:
- Second Mid Term test: Friday, June 13th, 2025, at 11:30 in Classroom 10 (Engineering Macroarea),
First Final Test: Friday, June 20th, 2025, at 11:00 in Classroom 10 (Engineering Macroarea),
- Second Final Test: Friday, July 18th, 2025, at 11:00 in Classroom C11 (Engineering Macroarea),
Students that decide to participate to the Second Mid Term are excluded from the participation to the First Final Test, as for the short time period that (in case of an insufficient qualification in the Second Mid Term) does not allow a proper recovery. Such students are of course admitted to the Second Final Test.
Students with a positive evaluation to the Mid Term 1 can also decide to register directly to the First Final test, and they resign in this way from an evaluation over the Mid Term tests.
All students applying for the Summer Session tests are invited to register to the respective exams at the Delphi pages.
Announcement:
Results of the First MidTerm Test held on April 28th, 2025.
Students are invited to write to the Course responsabile for accessing to their own written test.
Only admitted students are allowed to register to the second Mid Term (that will be held on June 13, 2025), while the other students are invited to register for Final Test 1 (that will be held on June 20, 2025). Please register to the two distinct exams at the Delphi pages.
Announcement: The First MidTerm Test of the course is scheduled on April 28th, 2025, and it will be held in Classroom C10, at 13:00. The test will focus on all the topics discussed during the lessons up to April 17th, 2025.
Students are invited to register to the exam at the Delphi pages.
News
Announcement: The lesson of Thursday April, 3rd will focus on a Lab exercise in the use of SVM and Kernel Learning software on different classification Tasks.
Proposed Exercises on HMM modeling.
-
NEW Course Lesson Timetable: The Course lessons will continue regularly since March 24, 2025, according to the following agenda that subsitute the previous one:
- MONDAY, h. 14:00-16:00
(Room C10 Classroom Building in the Engineering Macroarea)
- WEDNESDAY, h. 9:30-11:30
(Room B16 Classroom Building in the Engineering Macroarea)
- THURSDAY, h. 11:30-13:30
(Room B2 Classroom Building in the Engineering Macroarea)
-
As no change has been possible, the lesson of Thursday, March 20, 2025 will be held as usually in class C12 at 9:30.
Announcement OLD time table: The Course lessons will start regularly since March 3, 2025, according to the following timetable:
- MONDAY, h. 14:00-16:00
(Room C10 Classroom Building in the Engineering Macroarea)
- WEDNESDAY, h. 9:30-11:30
(Room B16 Classroom Building in the Engineering Macroarea)
- THURSDAY, h. 9:30-11:30
(Room C12 Classroom Building in the Engineering Macroarea)
-
The teams of the course under MS Teams is: "BASILI-8067802-DEEP_LEARNING".
Please check your membership on MS Teams. The course will start on March 3rd according to the following:
OLD Course TIMETABLE:
- MONDAY, h. 14:00-16:00
(Room C10 Classrooms Science Macroarea)
- WEDNESDAY, h. 9:30-11:30
(Room B16 Classrooms Engineering Macroarea)
- THURSDAY, h. 9:30-11:30
(Room C12 Classrooms Science Macroarea)
Please register for the Course on Delphi.
Teaching materials, such as papers PDFs or slides will be published on this page during the Course activities.
The Course refers to research and projects of the Semantics Analytics Group (SAG)
that makes research about Deep Learning and Natural Language Processing
in the design and engineering of Generative AI systems and complex AI Services.
These systems provide predictive and intelligent decision making functionalities
document search and interpretation, in the Knowledge Integration and Summarization in the
Intellligent Cybersecurity, in the Social Network Analysis and in all processes of Digital Transformation
in the industry or in specific domains, such as medicine or banking/fintech.
Active experimentations and projects are detailed at SAG Laboratory for Semantics Analytics,
that regularly financially supports Applied Research Internships or Laurea Thesis awards.
|
Course Contents
This is the preliminary Course syllabus updated across the Course and finalized and the end of the lessons.
Section I: Advanced Machine Learning and Deep Learning.
Deep Learning and Artificial Intelligence. Supervised methods.
Statistical Learning Theory: PAC learnability. Kernel-based Learning. Task dependent kernels.
Deep Learning with Neural Networks: Perceptron, Multilayer Percetrons, Deep Neural Networks. Image Processing with Deep Learning Architectures.
Section II: Neural Language Processing.
Neural Models for Language Processing. Language Models and Recurrent Networks. Attention-based Neural Networks. Transformers: BERT.
Introduction to the main software platforms for the development of DL software: PyTorch.
Large Language Models. Prompting and Instruction Tuning.
Neural models for Textual Inference: from few shot learning to prompting.
Section III: Deep Learning Applications.
Specialized Large Language Models in the Medicine, Biology, Nuclear Energy or Banking domains.
Visual Object Recognition, Automatic Image Captioning, Visual Question Answering.
Information Extraction. Fake News Detection.
Opinion Mining and Sentiment Analysis. Brand Reputation Analysis and Marketing.
|
Reference Text Books
- ML/DL - Introduction to Deep Neural Learning, Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, MIT Press, 2016.
- Neural Language Processing. Foundation Models for Natural Language Processing, Gerhard Paaß and Sven Giesselbach, Springer Nature, URL of the book.
- Probability and Computing. Introduction to Probability for Computing, Mor Harchol-Balter, Cambridge University Press, URL of the book, 2024.
-
ML. Pattern Recognition and Machine learning, C. Bishop. Springer. 2006.
-
Web IR - Bing Liu, Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. 2nd Edition, July 2011, Springer.
- Further Teaching Materials provided by the Teacher.
|
Lessons Slides
- In this section the slides of the different lessons and other teaching materials will be published.
-
Lesson 0: Deep Learning - a.a. 2024-25: Introduction: Course Organization and Exam Modalities.
-
Lesson 1: Introduction to Deep Learning in the perspective of Web Data Mining.
Lesson 2: Machine Learning Metrics and Evaluation (part I: metrics for Text Classification).
Complementary Materials (Non mandatory):
Lesson 3 : Probability and Learning: an introduction to Naive Bayes classifiers.
Lesson 4: Language Modeling - an Introduction to Hidden Markov Models for Sequence Labeling.
Complementary Materials (Non mandatory):
- Lesson 4a. Parameter Estimation for Language Modeling: the Baum-Welch algorithm.
Lesson 5:
(A gentle) Introduction to PAC learning and VC dimension. The slides used for the Course have been postedited from a kindly published version by Ethem Alpaydin, that you can find HERE.
Lesson 6: Support Vector Machines.
Lesson 7: Kernel Methods.
Lesson 8: Tree Kernels and NL Inference Tasks.
Section II - Introduction to Neural Networks and Deep Learning Architectures
Lesson 9: An Introduction to Neural Learning. The MultiLayer Perceptron: defining and training MLPs.
READINGS
- Nature Paper by Hinton and colleagues: Rumelhart, D. E., Hinton, G. E., and Williams, R. J., Learning representations by back-propagating errors.
- 1986 Rumelhart, D. E., Hinton, G. E., and McClelland, J. L.
A general framework for Parallel Distributed Processing, In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 45-76.
1986 Hinton, G. E., McClelland, J. L., and Rumelhart, D. E., Distributed representations., In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 77-109.
1986 Rumelhart, D. E., Hinton, G. E., and Williams, R. J., Learning internal representations by error propagation. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 318-362.
Exercises
- Lab 1 - Introduction to Keras: the XOR example.
- Lab 2a - A Linear classifier and a MLP for image classification over the MNIST dataset in Keras.
- Lab 2b - A Linear classifier and a MLP for image classification over the MNIST dataset in Pytorch.
Lesson 10 Deep Learning: Complex Architectures and Tasks: Convolutional Neural Networks
Laboratory Work:
References and resources:
Software packages for Deep Learning:
Lesson 11: Recurrent Neural Networks: architectures and applications
- BIO tagging with LSTMs: The task of BIO tagging and Entity Extraction, Training a Long-Short-Term-Memory Network in Keras on the Annotated Corpus for Named Entity Recognition using GMB (Groningen Meaning Bank) dataset, using the following file:
- NER Dataset
Lesson 12: Word embeddings as distributional semantic lexical representations.
Lesson 13 Neural Word Embeddings: between Language Modeling and Lexical Acquisition
Lesson 14 Attention: the introduction to Transformers
Lesson 15 Deep Learning: NLP tasks, Benchmarking Datasets and Evaluation
Lesson 16 Beyond Transformers: Decoder only Architectures.
From zero-shot and few-shot learning to large language model prompting.
Lesson 17 Beyond Transformers: Instruction Learning.
From zero-shot to Instruction Learning.
In-Context Learning for Relation Extraction using LLaMA 3 in the medical domain:
This lab introduces the task of relation extraction and demonstrates how to leverage LLaMA 3 through 0-Shot, One-Shot and Few-Shot prompting strategies.
The lesson materials and exercises are included in the following resources:
Fine-tuning of a decoder-only architecture (LLaMA) on multiple EVALITA 2023 tasks:
An applied exercise in instruction-tuning using Q-LoRA on several tasks from EVALITA.
All details and Colab links are included in the presentation.
Lesson 18 LLMs Trends: LoRA and RAG.
|
Link Utili
|
Laboratory Material and Exercises
- In this section Exercises and Solutions for the typical exam questions and tests will be published.
|