Multimedia and Deep Learning (seminar)

Students: In general I respond to emails within 2 workdays (Mon-Fri). If you do not get a response, please resend the email - Prof. Lew

It is necessary to register on the LML Course Manager:

     LML Course Manager, lcm.liacs.nl

Period: Tuesdays, from Feb. 7 to May 9
Time: 11am - 12:45pm
Place: Snellius 401

Organizer:

Prof. dr. Michael S. Lew, (Lecturer) email: lewmsk@gmail.com (Email to make an appointment)

Assistance and contributions given by:

Kai He, email: k.he@liacs.leidenuniv.nl

      Students: In emails, please use subject lines which start with MDL::

Recommended Prior Knowledge:

Because this course depends on knowledge in deep learning and visual imagery analysis, it is strongly recommended that students have successfully completed courses in machine learning. and

The student should be experienced in C and C++ programming (Python is also useful) and in image processing.

Registration

There are a limited number of students allowed in this course. Please contact Prof. Lew before enrolling.

Description and Goals:

Modern deep learning has its core and inspiration in the well known computer vision and multimedia problem of automatic image classification (what is in this picture?) by proposing deep convolutional neural networks (e.g. AlexNet and ImageNet) for image understanding. Since then, the fundamental ideas have been applied to a much wider set of data (also text, MRI, audio, etc.) with major successes in diverse fields.

This course examines the fundamental and important ideas in deep learning on multimedia data by using the scientific seminar format where in roughly half of the course there are scientific critical discussions on the important ideas, advances and scientific papers.

The discussions will cover the strengths and weaknesses, challenges and issues and future directions of computer vision and deep learning as methods of understanding diverse multimedia data.

The student must have moderate knowledge in C and C++ programming (Python is also useful) and should have an introductory level in image processing.

At the end of the course, the student should be able to

- understand the fundamental principles of multimedia and deep learning systems.
- analyze a multimedia deep learning system with regard to strengths and weaknesses and potential areas for improvements.
- have insight into traditional and state-of-the-art multimedia deep features.
- have insight into traditional and state-of-the-art multimedia deep learning algorithms.
- have insight into scientifically evaluating a multimedia and deep learning systems.
- have insight into the integration of intelligent algorithms into the analysis and retrieval process.
- have insight into the limits and challenges of modern multimedia and deep learning systems.
- develop and write scientific reports
- develop and give scientific presentations
- assess and evaluate the scientific credibility, rigor and reproducibility of deep learning and multimedia articles

Work-forms

- lectures
- seminar
- student discussions
- presentations
- homework and software assignments

Examination (for 6 ECs):

The final grade is composed of (1) 50% for Paper Presentation/Seminar, Class Participation & Questions & Assignments. (2) 50% for Final Project.

All programming assignments must work on Linux (Ubuntu 18 or 20 or the Ubuntu version in rooms 302/204)

For programming homework/assignments, the grading focuses on the software: source code, documentation, and how well the program works. Non-programming homework may be a blend of theoretical questions and/or scientific assessments of particular algorithms, where the main focus is not on programming, but on assessing and evaluating methods.

Assignments turned in late: grade penalty of -1 per 24 hours (1 day)

Source code for assignments must include instructions for compiling and execution in the machines in rooms 302, 303. This is necessary for grading/evaluating the work by the class organizers.

As this is a seminar, attendance is mandatory

University Leiden students do the work (see Examination above) including a Final Project for 6 ECTS.

For all assignments, students must clearly label which parts they wrote and submit their own solutions on LCM. Using tools such as ChatGPT are not allowed in this course.

- Note that if writing source code is required, the problem will explicitly mention it. Otherwise, for example in calculations in experiments, it is fine to either do it manually or write code for it.

Final Project

In principle, this is meant to be a scientific toned study between at least 2 competing algorithms for the same task such as VGG vs Resnet in image classification. The goal is to gain insights into the strengths and weaknesses of each algorithm and also the challenges in performing quantitative fair comparisons.

Suggestions for class presentations

- Introduce the problem and main idea - Explain the motivation for the paper
- Cover the main points of the algorithm
- Make sure you have a slide which clearly states the main contribution of the paper at the end
- There should be a little humor in the presentation to make it interesting for the audience
- USB Stick - Bring a backup PDF on a USB Stick

Schedule (TENTATIVE - depends on number of students in class):

2022-05-16 - Final Project Presentations and Submissions Due