Deep Learning (Data Scientist) 2 — Questions and Answers
Question 1: A data scientist notices training loss decreasing while validation loss starts rising after epoch 15. What is the most appropriate response?
- Apply early stopping or regularization (Correct answer)
- Increase the learning rate
- Add more hidden layers
- Remove the validation set
Correct answer: Apply early stopping or regularization
Diverging training and validation loss indicates overfitting, which early stopping or regularization directly addresses.
Question 2: Which technique randomly deactivates a fraction of neurons during training to reduce overfitting?
- Dropout (Correct answer)
- Batch normalization
- Gradient clipping
- Weight sharing
Correct answer: Dropout
Dropout randomly zeroes out neurons during training, forcing the network to learn redundant representations.
Question 3: Why does batch normalization typically allow the use of higher learning rates?
- It stabilizes the distribution of layer inputs during training (Correct answer)
- It reduces the number of trainable parameters
- It eliminates the need for activation functions
- It removes the bias terms from each layer
Correct answer: It stabilizes the distribution of layer inputs during training
By normalizing layer inputs, batch normalization smooths the optimization landscape, permitting larger learning rates without divergence.
Question 4: A model for credit risk uses a sigmoid output. The predicted probabilities cluster near 0.5 despite clear class separation in the data. Which is the most likely cause?
- The model is underfitting due to insufficient capacity or training (Correct answer)
- The dataset has too many features
- The sigmoid function is saturating at its extremes
- The batch size is too small
Correct answer: The model is underfitting due to insufficient capacity or training
Predictions stuck near 0.5 on separable data signal underfitting, since a well-trained model would push probabilities toward the extremes.
Question 5: Which loss function is most appropriate for a multi-class classification network with a softmax output layer?
- Categorical cross-entropy (Correct answer)
- Mean squared error
- Hinge loss
- Huber loss
Correct answer: Categorical cross-entropy
Categorical cross-entropy directly measures the divergence between the softmax probability distribution and the one-hot target.
Question 6: In transfer learning for image classification with a small dataset, which strategy is generally recommended first?
- Freeze the pretrained convolutional base and train only a new classifier head (Correct answer)
- Train the entire network from random initialization
- Fine-tune all layers with a high learning rate
- Remove all convolutional layers and use dense layers only
Correct answer: Freeze the pretrained convolutional base and train only a new classifier head
Freezing the pretrained base preserves learned features while the small dataset trains only the lightweight classifier, reducing overfitting risk.
Question 7: What problem do residual (skip) connections in ResNet primarily solve?
- Degradation of accuracy in very deep networks due to difficult gradient flow (Correct answer)
- Excessive memory consumption during inference
- Class imbalance in the training data
- Slow data loading during training
Correct answer: Degradation of accuracy in very deep networks due to difficult gradient flow
Skip connections let gradients flow directly through identity paths, enabling very deep networks to train without degradation.
Deep Learning (Data Scientist) 2
A data scientist notices training loss decreasing while validation loss starts rising after epoch 15.
What is the most appropriate response?