Esci dai Frame

	Web Mining e Retrieval - Deep Learning (a.a. 2022/23) Secondo Semestre
Elenco dei File nel deposito
Sommario Contenuti 1.Novita' 2.Programma del Corso 3.Testi di Riferimento 4.Link Utili 5.Diapositive delle lezioni 6.Progetti ed Esercizi Proposti
Novita' Results of the Second Final Test on June 18th, 2023: Students that are evaluated with a final score higher than 18/30 are invited to communicate their acceptance via mail, to the Course responsible mail: basili@info.uniroma2.it. Results of the Second MID Term and the First Final Test on June 21st, 2023: Students that are evaluated with a final score higher than 18/30 are invited to communicate their acceptance via mail, to the Course responsible mail: basili@info.uniroma2.it. Students that need to carry out a project to collect their 9 CFU should contact the Course responsible for selecting a suitable topic. Before accepting the evaluation, every student can access to the test outcomes, in the Course Responsible's Office, according to the following agenda: FRIDAY July 7th, 2023, h. 15:00-16:00 MONDAY July 10th, 2023, h. 16:00-17:00 FULL Summer Exams Agenda: The Summer Session will include the following exams at the following dates: The Second Mid Term and the First Final Exam Test will be held on Wednesday June 21st, at h. 10:00-13:00 in Room C6, (Engineering Area) The Second Final Exam Test will be held on July 18th, at h. 10:00-13:00 in Room 10 (Engineering Area). Students need to register to the different Tests through the Delphi platform. Please, for the June 21st test, specify if you register for the Mid Term 2 or for the First Final Exam. Results of the First MID Term Test on April 26th, 2023: All students that are admitted with a temporary evaluation higher than 18/30 are invited to register to the Mid Term 2 Test through the Delphi platform. !!WARNING: First MID Term DATE CHANGE!!: The First Mid Term Test will be held on Wednesday April 26th, during the morning lesson slot (h. 9:00-11:30 in Room B14, Engineering Area) and NOT on the 27th, as previously announced. The Test will still cover topics from ALL the lessons from 1 to 8. Students need to register to the Test through the Delphi platform. NEWS: All students are kindly requested to install Anaconda, a distribution of the Python and R programming languages for scientific computing, that makes available the different main packages useful for the Lab: Jupyter Notebook Python, as "Anaconda is a completely free Python distribution. It includes more than 400 of the most popular Python packages for science, math, engineering, and data analysis SciKit Learn Please locate the Anaconda installation executable you need to download for your PC at the following address: https://www.anaconda.com/products/distribution. Then Run the installation and follow the instructions. The slides of the two lessons are uploaded as follows: Introduction to Evaluation Metrics and Practices in Machine Learning Introduction to Jupyter Notebooks. IRIS Data set Classification through kNN: Exercise Notebook (Save and run it through Jupyter NoteBook) ANNUNCIO: Il Corso avra' inizio a partire dal 6 Marzo 2023, secondo il seguente orario: LUNEDI', h. 14:00-16:00 (Aula 13 Edifici Aule Macroarea di Scienze) MERCOLEDI', h. 9:30-11:30 (Aula B14 Edifici Aule Nuove della Macroarea di Ingegneria) GIOVEDI'', h. 11:30-13:30 (Aula 13 Edifici Aule Macroarea di Scienze) The teams of the course under MS Teams is: "BASILI-8038971-WM&R-2022_23". Please check your membership on MS Teams. The course will start on March 7 according to the following: Course TIMETABLE: LUNEDI', h. 14:00-16:00 (Aula 13 Classrooms Science Macroarea) MERCOLEDI', h. 9:30-11:30 (Aula B14 Classrooms Engineering Macroarea) GIOVEDI'', h. 11:30-13:30 (Aula 13 Classrooms Science Macroarea) Gli studenti che intendono seguire il Corso sono pregati di registrarsi ad esso, accedendo al sito Delphi. Le diapositive delle lezioni saranno pubblicate durante il ciclo delle lezioni su queste pagine. Il Corso insiste sulle ricerche ed i progetti innovativi del Semantics Analytics Group (SAG), che si occupa di Deep Learning e Natural Language Processing nella progettazione ed ingegnerizzazione di Sistemi Software Avanzati di Intelligenza Artificiale. Tali sistemi sono applicati a processi predittivi nella interpretazione e ricerca di documenti, nella sicurezza in rete, nella analisi dei Social Network e nei processi di Digital Transformation nell'Industria e nella Medicina. Sono attive alcune sperimentazioni e progetti presso il SAG Laboratory for Semantics Analytics, da cui sono emanate annualmente alcune Borse di Studio e Premi di Laurea. Sara' possibile discutere in dettaglio le diverse Tesi con il coordinatore di SAG, prof. Roberto BASILI, o con il responsabile tecnico del Laboratorio, prof. Danilo CROCE. L'orario di ricevimento, diverso da quello dei Corsi, e' da concordare con i docenti via e-mail.
Programma Segue il programma preliminare del Corso che sara' messo a punto ed finalizzato al termine delle lezioni del Corso. Section I: Machine Learning and Deep Learning. Deep Learning and Artificial Intelligence. Supervised methods. Probabilistic and Generative Methods. Hidden Markov Models. Statistical Learning Theory: PAC learnability. Kernel-based Learning. Polynomial and Radial Basis Function Kernels. String and Tree kernels. Semantic kernels. Neural Modeling: Perceptron, Multilayer Percetrons, Deep Neural Networks. Language Models and Recurrent Networks. Attention-based Neural Networks. Transformers: BERT, GPT models. 0-Shot Learning. Introduction to the main software platforms for the development of ML software: Weka, SciKit, PyTorch, KeLP. Section II: Deep Networks for Natural Language Processing and Social Media Analytics. Neural Language Processing. Named Entity Recognition. Statistical and Neural parsing.Shallow Semantic Parsing: Semantic role labelling. Neural models for Textual Inference: from few shot learning to prompting. Information Extraction for Business Modeling. Question Answering Systems. Wikipedia-based knowledge Acquisition. Social Web Analysis. Opinion Mining and Sentiment Analysis. ML Application for the Digital Transformation: Sentiment Analysis, Brand reputation and real Time Marketing. Social Media Analytics: fake news counteraction. Ethical dimensions of Neural Decision Making. .
Testi di Riferimento ML/DL - Introduction to Deep Neural Learning, Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, MIT Press, 2016. ML - Pattern Recognition and Machine learning, C. Bishop. Springer. 2006. ML ed IR - Introduction to Information Retrieval , Christopher D. Manning, Prabhakar Raghavan and Hinrich Schutze, Cambridge University Press. 2008. Find the book Home page HERE. ML ed IR - Automatic Text Categorization: from Information Retrieval to Support Vector Learning, Roberto Basili, Alessandro Moschitti, ARACNE Editore, 2005. Web IR - Bing Liu, Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. 2nd Edition, July 2011, Springer. Dispense fornite dal docente
Lezioni (Lessons Slide) In questa sezione verranno pubblicate le diapositive delle lezioni e altri materiali didattici. Lesson 0: WMR - a.a. 2022-23: Introduction: Course Organization and Exam Modalities. Lesson 1: Introduction to Web Mining & Retrieval. Some slides of Lesson 1 refer to the discussion of the link: "A visual introduction to ML", slide 24). Lesson 1.1: Machine Learning: target problems and major paradigms. Lesson 2: Introduction to Geometrical models of text classification: Profile-based classification, Rocchio and k-NN. Lesson 3: Machine Learning Metrics and Evaluation (part I: metrics for Text Classification). Lesson 4 : Probability and Learning: an introduction to Naive Bayes classifiers. Lesson 5: (A gentle) Introduction to PAC learning and VC dimension. The slides used for the Course have been postedited from a kindly published version by Ethem Alpaydin, that you can find HERE. C. Burges's Tutorial on SVM and VC dimension. D. Haussler discussion of PAC Learning, 1999. Valiant L. G. A Theory of the Learnable, Communications of the ACM, Volume 27 Issue 11, Nov. 1984 Pages 1134-1142 . Lesson 6: Support Vector Machines. An animated Perceptron. Dan Klein's tutorial on Lagrange methods for the SVM optimization problem. Lesson 7 Kernel-based learning - first part: from kernel definition to String Kernels. Lesson 8: Introduction to language modeling: from Information Retrieval tasks to Natural Language Processing. Short Bibliography as an Introduction to NLP and Text Mining: Context-free grammars: "Linguaggi, Modelli, Complessit`a" G. Ausiello, G. Gambosi, F. D'Amore, A technical introduction in Italian. "On Certain Formal Properties of Grammars" (N. Chomsky, 1959) Noam Chomsky's "Syntactic Structures", the 1957 book Lesson 9: Language Modeling - an Introduction to Hidden Markov Models for Sequence Labeling. Complementary Materials: Lesson 9a. Parameter Estimation for Language Modeling: the Baum-Welch algorithm. Lesson 9b. Parameter Estimation and Rare Phenomena in Language Modeling. Lesson 10: Introduction to Neural Networks and Deep Learning Architectures: a mini Course. Lesson 10.1 An Introduction to Neural Learning. The MultiLayer Perceptron: defining and training MLPs. Lab 1 - Introduzione a Keras: the XOR example. Lab 2a - A Linear classifier and a MLP for image classification over the MNIST dataset in Keras. Lab 2b - A Linear classifier and a MLP for image classification over the MNIST dataset in Pytorch. Lesson 10.2 Deep Learning: Complex Architectures and Tasks: Convolutional Neural Networks References and resources: Gradient Descent and Perceptron training, "A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm", Quoc V. Le (Google), (fino a sezione 6 compresa) Backpropagation on NNs, "Sparse autoencoder", from the lessons of Andrew Ng (Stanford University, USA), (fino a pagina 12) Advanced Architectures, "A Tutorial on Deep Learning, Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks", by Quoc V. Le (Google Brain) Deep Learning, "Learning Deep Architectures for AI", Yoshua Bengio Dept. IRO, Universite' de Montreal, CA. An interesting in-depth analysis on CNN Software packages for Deep Learning: TensorFlow Anaconda Torch Keras Tensorflow Pytorch Convolutional Neural Networks for Visual Recognition: CNNs for Visual Recognition on GitHub Lesson 10.3 Recurrent Neural Networks, Encoder and Decoder Networks Lesson 10.4. Word embeddings as distributional semantic lexical representations. Lesson 10.5 Neural Word Embeddings: between Language Modeling and Lexical Acquisition Lesson 10.6 Attention: the introduction to Transformers Lesson 10.7 Beyond Transformers: from zero-shot and few-shot learning to large language model prompting Lezione 11 Short Introduction to Opinion Mining. See also: Tutorial on Opinion Mining (by B. Liu) Book Chapter on Opinion Mining and Sentiment Analysis (by Bo Pang and Lillian Lee) Lezione 12 A case study: Sentiment Analysis in Twitter. Introduction to the Second MidTerm test : only questions and topics, no solution..