Machine Learning Neural Networks 3 — Questions and Answers
Question 1: What is the primary purpose of the softmax function in the output layer of a classifier?
- To bound outputs between -1 and 1
- To convert raw logits into a probability distribution summing to 1 (Correct answer)
- To apply non-linearity to hidden layers
- To normalize weights across neurons
Correct answer: To convert raw logits into a probability distribution summing to 1
Softmax exponentiates each logit and divides by the sum, converting raw scores into probabilities that sum to 1 across all classes.
Question 2: What distinguishes a generative adversarial network (GAN) from a standard neural network?
- GANs use unsupervised clustering instead of backpropagation
- GANs consist of two networks — a generator and discriminator — trained adversarially (Correct answer)
- GANs replace gradient descent with evolutionary algorithms
- GANs use only convolutional layers without fully connected layers
Correct answer: GANs consist of two networks — a generator and discriminator — trained adversarially
A GAN pits a generator that creates fake data against a discriminator that tries to distinguish real from fake, training both simultaneously.
Question 3: In neural networks, what is a hyperparameter?
- A weight learned during backpropagation
- A configuration setting chosen before training begins (Correct answer)
- The bias added to each neuron's activation
- The gradient computed at each layer
Correct answer: A configuration setting chosen before training begins
Hyperparameters like learning rate, batch size, and number of layers are set before training and are not updated by the optimizer.
Question 4: What does the term 'epoch' refer to in neural network training?
- A single forward pass through one batch
- One complete pass through the entire training dataset (Correct answer)
- The total number of weight updates performed
- The time taken to compute one gradient step
Correct answer: One complete pass through the entire training dataset
An epoch is one full cycle where the model has seen every training example exactly once.
Question 5: Which optimizer adapts the learning rate for each parameter individually using estimates of first and second moments of gradients?
- SGD with momentum
- RMSprop
- Adam (Correct answer)
- Adagrad
Correct answer: Adam
Adam (Adaptive Moment Estimation) maintains per-parameter adaptive learning rates using exponentially decaying averages of past gradients and squared gradients.
Question 6: What is weight initialization and why does it matter in deep networks?
- Setting weights to their final values after training
- Choosing initial weight values to enable stable gradient flow from the start (Correct answer)
- Resetting weights after each epoch to avoid overfitting
- Copying weights from a pretrained model
Correct answer: Choosing initial weight values to enable stable gradient flow from the start
Poor initialization (e.g., all zeros) can cause symmetry problems or vanishing/exploding gradients, preventing the network from learning.
Question 7: In transfer learning, what does 'fine-tuning' mean?
- Training a model from scratch on a new task
- Freezing all pretrained layers and training only a new head
- Continuing training of a pretrained model on a new dataset with a small learning rate (Correct answer)
- Pruning unnecessary neurons from a pretrained model
Correct answer: Continuing training of a pretrained model on a new dataset with a small learning rate
Fine-tuning starts with pretrained weights and continues updating them on a target task, typically with a lower learning rate to preserve learned features.
Machine Learning Neural Networks 3
What is the primary purpose of the softmax function in the output layer of a classifier?