Deep Generative Models / Summer 2026
Updates
- New Lecture is up: Lecture 3 - Part 3/3: Generative Learning and Naive Bayes
- New Lecture is up: Lecture 3 - Part 2/3: Discriminative vs Generative Learning
- New Lecture is up: Lecture 3 - Part 1/3: Fundamentals of Data Generation
- New Assignment released: [Project Proposal]
-
Erik Saarenvirta from Google will give a Guest lecture on May 26.
- New Assignment released: [Assignment #1 - Language Modeling]
- New Lecture is up: Lecture 2 - Part 2/2: Large Language Models
For the Quercus page of the course please click here
Guest Lecture by Erik Saarenvirta
On May 26, we will hve a guest lecture by Erik Saarenvirta from Google.
Title:
Building AI Supercomputers on Google Cloud
Abstract:
Discover how modern AI supercomputers are built from the silicon up. This lecture explores the architecture of Google’s TPUs, how the JAX compiler seamlessly scales math across thousands of chips, and the Kubernetes orchestration required to manage it all.
Bio:
Erik Saarenvirta is a Solutions Architect at Google specializing in high-performance computing (HPC), where he helps organizations design and scale their production AI.infrastructure workloads
Course Description
This course aims to build a good understanding on Generative Models, the state-of-the-art architectures and their applications. The course in designed in two major parts, where in the first part the basic idea of data generation is introduced, and some mathematical foundations are provided. In this respect, the course looks at Large Language Models as an initial generative model and discusses some advancement in this respect. The second part of the course gets into the three major approaches for designing a generative model: namely, generative adversarial networks (GANs), variational autoencoders (VAEs), and diffusion models. The course finally goes through some recent advancements in generative AI. To follow the contents of this course, the students need to have a good understanding of machine learning models, particularly neural networks. Knowledge in more advanced topics like unsupervised learning methods, or newer architectures will be given in the course throughout the semester. A primary course in Machine Learning has hence been considered as the prerequisite. It is expected that the second part of the course dominates, i.e., approximately half of lectures will be on deep generative models.
Time and Place
Lectures
Lectures start on May 5, 2026.
| Day | Time | Place |
| Tuesdays | 1 - 4 PM | MP-102 - McLennan Physical Laboratories |
Tutorials
Tutorials sessions start on May 14, 2026.
| Day | Time | Place |
| Thursdays | 1 - 2 PM | BA1160 - Bahen Centre for Information Technology |
Course Office Hours
| Day | Time |
| Tuesdays | 4:00 - 5:00 PM |
Previous Offerings
Course Syllabus
The course consists of three major parts: in Part I, we start with language models. Note that we do not get into the details from the natural language processing viewpoint, and mainly get the idea of how we can build a generative model. We will then formally define the problem of data generation and go through the key ideas for building a generative model, i.e., sampling from a data distribution, in Part II. Part III goes briefly through the recent advancements. Below is the detailed syllabus.
Part I: Initial Trial to Generate Data
- Generating text
- Review: Sequence-to-sequence models
- Language model
- Large Language Models
- Looking into some LLMs
- Using LLMs in practice
- Training
- Fine-tuning
- Prompting
- Stochastic viewpoint on data generation
- Looking at data as random process
- Data space versus dataset
- Data distribution
- Data generation by sampling from data distribution
Part II: Deep Generative Models
- Learning explicitly the data distribution
- Approximating data distribution via learnable models
- Training via maximum likelihood
- Some early examples
- Autoregressive Models
- Flow-based Generative Models
- GANs: Sampling data distribution implicitly via adversarial networks
- Generative adversarial networks (GANs)
- Minmax game
- Implicit learning of data distribution
- Interpreting minmax problem as divergence minimization
- Wasserstein GANs and recent advancements
- Replacing KL divergence with Earth Mover (EM)
- Impacts of EM minimization on stability
- VAEs: Sampling data distribution via variational inference
- Review: Autoencoders and latent space
- The concept of latent space
- Data generation by sampling from latent space
- Variational inference
- Variational Autoencoders
- Autoencoders with variational latent space
- Data generation by sampling from variational latent space
- Diffusion Models: Sampling data distribution via Langevin dynamics
- Langevin dynamics and the diffusion process
- Forward and reverse processes
- Forward process – Noising procedure
- Reverse process – Denoising procedure
- Score-based diffusion models
- Probabilistic diffusion models
Part III: Recent Advancements
- Multimodal Models
- Conditional generative models
Course Evaluation
The learning procedure consists of three components:
| Component | Grade % | |
| Assignments | 40% | 5 assignment sets |
| Exam | 30% | exam |
| Project | 30% | open-ended |
Assignment
This is the most important part: there is no way to learn a topic like Generative Models unless we get our hands dirty and implement whatever we learn. Roughly speaking, the first one goes through basics and make you play around with language models. The second one goes through go through different ideas for explicit data generation, and the last three ones go through GAN, VAE and Diffusion. The assignments include both analytic and programming tasks and count roughly for 40% of your final mark.
Exam
We have two midterm exams in this semester. Each exam will ask two questions similar to assignments and practice questions shared with you. The exam evaluates the understanding of fundamental concepts. This exam will comprise 30% of the total mark.
Course Project
The course project procedure is illustrated in a seperate page. In a nutshell, the project will be in groups of 3. Each group selects a topic and work on the project through the semester and present its final result at the end in a report and a short presentation. The course project is the key evaluator and comprises 30% of the course work.