Organisation

Time only SS
Time Tuesday, 11.45h-13.15h
Room 137
Credits 2 SWS / 5 ECTS
Exam Paper and Presentation

Announcements

  • First lesson in term SS 20: 17.03.2020

Prerequisites for participation

  • Successful completion of the CSM Machine Learning Lecture

Resources and Presentations

Structure

Theory Part

Based on the theory and algorithms, which have been taught in the Machine Learning Lecture, this seminar focuses on very recent Deep Learning algorithms for

  • Text
  • Images

Concerning text, the currently hottest topic in NLP are deep pretrained models, which provide contextual embeddings and can be fine-tuned applied for many NLP downstream tasks, such as Named Entity Recognition, Question-Answering, Automatic Text Generation, Natural Language Inference, Text Classification, etc. BERT, GPT-2 or XLNET are the most prominent representatives of this class of transformer models. In order to understand the theory and the applications of these models this lecture provides key-insights in

  • modelling of texts, words and characters (embeddings)
  • attention and self attention in neural networks
  • transformer models
  • BERT, GPT-2, XLNET

Concerning image, this lecture focuses on Generative Adversarial Networks (GANs). The theory and architecture of the most important GANs are highlighted, and their applications, e.g.

  • automatic generation of photorealistic images,
  • image transformation and style transfer,
  • image modification, e.g. changing the emotion in a human’s face
  • super resolution

are demonstrated.

This part of the lecture will be stretched over the approximately first 5 weeks of the term.

Group Work Part

In the second part of the term, student-groups select one current topic of Deep Learning and analyse this topic in depth. A corresponding paper has to be prepared. The topics are presented by the student-groups at the end of the term.

More information on this part:

  1. There should be 2-4 students per group
  2. Each group studies the selected topic during the term. This study includes the following:
    1. Literarure Research:
      1. Determine the most important papers and approaches for the selected topic
      2. Which approaches do they apply? How do they differ and how are they related to each other?
      3. What is the impact of the paper (on other scientific work or applications)
      4. Presentation of literature-research
    2. Select a main paper for the topic:
      1. Understand and investigate the approach of the selected paper in detail
      2. Presentation of the technique (in detail) at the end of this course
    3. Reimplement the experiment of the selected paper and reconstruct its result
    4. Suggest modificications and improvements, implement and validate/evaluate them
    5. Final Presentation

List of potential topics to investigate (from SS 2019)

Deep Neural Networks for Language Understanding:

Explainable AI and Causality

Style Transfer Networks

Low-Resource-Learning / Transfer Learning / Active Learning

AlphaGo / AlphaZero / AlphaStar