Machine Learning Deep Learning 4 — Questions and Answers
Question 1: In a generative adversarial network (GAN), what is the role of the discriminator?
- To generate realistic synthetic samples
- To distinguish between real and generated samples (Correct answer)
- To encode input images into a latent space
- To decode latent vectors into images
Correct answer: To distinguish between real and generated samples
The discriminator is a classifier trained to tell apart real data from fake data produced by the generator, providing adversarial feedback.
Question 2: What does the term 'epoch' mean in neural network training?
- One forward pass through a single training sample
- One complete pass through the entire training dataset (Correct answer)
- One update of the model weights
- One iteration of hyperparameter search
Correct answer: One complete pass through the entire training dataset
An epoch is completed when the model has seen every sample in the training set exactly once, typically composed of multiple mini-batch updates.
Question 3: What is the primary difference between L1 and L2 regularization in deep learning?
- L1 penalizes the sum of squared weights; L2 penalizes the sum of absolute weights
- L1 penalizes the sum of absolute weights, promoting sparsity; L2 penalizes squared weights (Correct answer)
- L1 is applied to activations; L2 is applied to gradients
- L1 increases model depth; L2 increases model width
Correct answer: L1 penalizes the sum of absolute weights, promoting sparsity; L2 penalizes squared weights
L1 regularization adds |w| to the loss, driving some weights to exactly zero (sparsity), while L2 adds w² and shrinks weights uniformly.
Question 4: Which optimizer adapts the learning rate for each parameter based on past gradients and is commonly used in training transformers?
- SGD with momentum
- Adam (Correct answer)
- Adagrad
- RMSProp
Correct answer: Adam
Adam combines momentum and RMSProp, maintaining adaptive per-parameter learning rates using first and second moment estimates of gradients.
Question 5: What is a hyperparameter in the context of training a deep neural network?
- A weight learned during backpropagation
- A configuration value set before training that controls the learning process (Correct answer)
- The output of the final activation function
- A parameter stored in the model's output layer
Correct answer: A configuration value set before training that controls the learning process
Hyperparameters like learning rate, batch size, and number of layers are chosen before training and are not updated by the optimizer.
Question 6: What is the exploding gradient problem and how is it commonly addressed?
- Gradients become zero; addressed by adding more layers
- Gradients grow exponentially during backpropagation; addressed by gradient clipping (Correct answer)
- Weights grow unbounded during inference; addressed by dropout
- Loss becomes NaN due to bad data; addressed by normalization
Correct answer: Gradients grow exponentially during backpropagation; addressed by gradient clipping
Exploding gradients cause unstable updates; gradient clipping caps the gradient norm to a maximum threshold during training.
Question 7: What does 'fine-tuning' mean when adapting a pretrained deep learning model?
- Training the model from scratch on a new dataset
- Continuing to train some or all layers of a pretrained model on a new task (Correct answer)
- Pruning unnecessary neurons from the model
- Quantizing the model weights for deployment
Correct answer: Continuing to train some or all layers of a pretrained model on a new task
Fine-tuning updates the pretrained weights on new task data, often with a small learning rate, to adapt learned representations without losing general knowledge.
Machine Learning Deep Learning 4
In a generative adversarial network (GAN), what is the role of the discriminator?