Deep Learning Generative Models 1 โ Questions and Answers
Question 1: What are the two main components of a Generative Adversarial Network (GAN)?
- Encoder and decoder
- Generator and discriminator trained in opposition to each other (Correct answer)
- Policy network and value network
- Inference network and sampling network
Correct answer: Generator and discriminator trained in opposition to each other
A GAN consists of a generator that creates fake samples and a discriminator that distinguishes real from fake, with each trained adversarially to improve the other.
Question 2: What is mode collapse in GAN training?
- The discriminator loss collapsing to zero
- The generator producing only a few distinct outputs rather than the full data distribution (Correct answer)
- The GAN failing to converge during early training
- The discriminator ignoring generated samples
Correct answer: The generator producing only a few distinct outputs rather than the full data distribution
Mode collapse occurs when the generator learns to produce a limited variety of outputs that fool the discriminator, ignoring most modes of the real data distribution.
Question 3: What is a Variational Autoencoder (VAE) and how does it differ from a standard autoencoder?
- A VAE is identical to a standard autoencoder but uses convolutional layers
- A VAE encodes inputs as probability distributions in latent space rather than fixed vectors, enabling principled sampling and generation (Correct answer)
- A VAE uses adversarial training instead of reconstruction loss
- A VAE has no decoder, only an encoder
Correct answer: A VAE encodes inputs as probability distributions in latent space rather than fixed vectors, enabling principled sampling and generation
A VAE learns a probabilistic encoder mapping inputs to distributions (mean + variance) and uses the reparameterization trick to enable backpropagation through the sampling step.
Question 4: What does the reparameterization trick in VAEs enable?
- Using a different activation function in the encoder
- Backpropagation through the stochastic sampling step by expressing samples as a deterministic function of parameters and a noise variable (Correct answer)
- Increasing the dimensionality of the latent space during training
- Replacing the KL divergence term with cross-entropy
Correct answer: Backpropagation through the stochastic sampling step by expressing samples as a deterministic function of parameters and a noise variable
The reparameterization trick writes z = ฮผ + ฯยทฮต (ฮต ~ N(0,1)), making the sample a differentiable function of ฮผ and ฯ so gradients can flow through sampling.
Question 5: What is a diffusion model in the context of generative deep learning?
- A model that diffuses information across spatial dimensions using convolutions
- A generative model that learns to reverse a gradual noising process to generate data from pure noise (Correct answer)
- A model that applies attention over a noise schedule to produce embeddings
- A GAN variant using spectral normalization for stability
Correct answer: A generative model that learns to reverse a gradual noising process to generate data from pure noise
Diffusion models define a forward process that gradually adds noise to data and train a neural network to denoise, learning to generate data by iteratively denoising Gaussian noise.
Question 6: What loss function does a standard GAN generator minimize?
- Reconstruction loss between generated and real images
- The negative log-probability of the discriminator classifying generated samples as real (Correct answer)
- Cross-entropy between generated distribution and target distribution
- Mean squared error between latent codes and output images
Correct answer: The negative log-probability of the discriminator classifying generated samples as real
The generator is trained to maximize the discriminator's probability of labeling generated samples as real, equivalent to minimizing the negative log of that probability.
Deep Learning Generative Models 1
What are the two main components of a Generative Adversarial Network (GAN)?