Web Search and Data Mining

Course Description

Web Search and Data Mining (WiSDoM) is an area that aims at extracting knowledge from the largest source of information created by humans: The Web! Throughout this course we will see how this extracted knowledge can solve complex tasks with advanced Computer Vision, Natural Language and Information Retrieval algorithms. The main topics of this course are:

• Text and visual data representation
• Billion scale text and image search
• Visual question answering
• Multimodal conversational assistants
• Recommender systems

This course includes intensive hands-on laboratories where key CV, NLP and IR algorithms are examined.

Objectives

• Learn what is an information embedding.
• Learn the semantic associations between visual data, natural language data and user data.
• Learn how to relate user information needs to actionable data.
• Learn how to do a critical analysis of experimental results.
• Develop autonomous and creative problem solving skills.

Grading

• Exam (40%)
• Project (45% = 15% phase 1 + 20% phase 2 + 10% phase 3)
• Project originality (15%)

Schedule

• 09/mar/21 Introduction
• 16/mar/22 Web document categorization
• 23/mar/22 Word embeddings
• 30/mar/22 Transformer (encoder)
• 06/abr/22 Billion scale indexing
• 13/abr/22 Visual search
• 20/abr/22 Vision-and-language models
• 27/abr/22 ExpoFCT
• 04/mai/22 Multimodal conversational assistants
• 11/mai/22 Pre-trainning and fine-tunning
• 18/mai/22 Transformer (decoder)
• 25/mai/22 Recommender systems
• 01/jun/22 HuggingFace invited lecture
• 08/jun/22 Project discussion / Test

Tutorials

We suggest you to use the account in the lab cluster. However, if you would like to have your own setup, you can follow this guide:

Lecturers

Joao Magalhaes ([email protected] - remove the ‘x’ character to send an email)