Inhalte

In this lecture, we will cover the most recent approaches and principles at the frontier of learning and applying of foundation models in language, vision (image generation, image segmentation, image understanding) as well as multimodal (=multimedia) data processing, including

1. learning vision foundation models from natural language supervision, with applications to open-vocabulary image classification and retrieval, object detection, segmentation, and multimodal understanding;
2. learning vision foundation models via masked image modeling, with its extensions to multimodal pre-training
3. vision foundation model architecture design with transformer and beyond.

Übungen

Es erscheint wöchentlich ein Übungsblatt zu den behandelten Vorlesungsinhalten. Jedes Übungsblatt wird in der Globalübung der folgenden Woche besprochen. Es gibt keine Abgabe / Korrektur von Übungsblättern.

Turnus

Die Vorlesung wird in jedem Sommersemester gelesen.