Deep Learning Generative Models 3

Question 1

What is the ELBO (Evidence Lower BOund) in variational inference used in VAEs?

Accepted Answer

A lower bound on the log-likelihood of data, maximized to train the VAE by balancing reconstruction and KL terms

Answer

An upper bound on model complexity

Answer

The exact log-likelihood when the posterior is Gaussian

Answer

A metric for evaluating generated image quality

Question 2

What is the Fréchet Inception Distance (FID) used to measure?

Accepted Answer

The quality and diversity of generated images by comparing statistics of generated and real image feature distributions

Answer

Training speed of generative models in FLOPS

Answer

The distance between two points in latent space

Answer

Discriminator accuracy in GAN training

Question 3

What is a denoising score matching objective used to train diffusion models?

Accepted Answer

Training a neural network to predict the noise added at each diffusion step, enabling iterative denoising

Answer

Training the model to predict the clean image directly from noisy input

Answer

Matching discriminator scores between real and generated samples

Answer

Computing cosine similarity between encoded image features

Question 4

What is a conditional diffusion model like Stable Diffusion conditioned on?

Accepted Answer

Text embeddings from a language model like CLIP or a text encoder, provided via cross-attention during denoising

Answer

The pixel values of a seed image only

Answer

A categorical class label provided as a one-hot vector

Answer

Gradient information from a discriminator network

Question 5

What distinguishes a generative model from a discriminative model?

Accepted Answer

Generative models learn the joint distribution p(x,y) and can generate samples; discriminative models learn p(y|x) for classification only

Answer

Generative models always use unsupervised learning; discriminative models always use supervised learning

Answer

Generative models are always deeper; discriminative models are always shallower

Answer

Generative models produce text; discriminative models produce images

Question 6

What is the purpose of classifier-free guidance in diffusion models?

Accepted Answer

Amplifying the conditioning signal by interpolating between conditional and unconditional score estimates, improving adherence to prompts at the cost of diversity

Answer

Removing the text conditioning pathway to generate unconditional samples

Answer

Using a separate classifier to filter out low-quality generated samples

Answer

Guiding the diffusion process using a pretrained image classifier's gradients