Neural Network Regularization and Overfitting

Question 1

What does overfitting in a neural network mean?

Accepted Answer

The model performs well on training data but poorly on unseen data

Answer

The model performs well on both training and test data

Answer

The model fails to learn the training data patterns

Answer

The model has too few parameters to fit the data

Question 2

Which regularization technique adds the sum of absolute values of weights to the loss function?

Accepted Answer

L1 regularization

Answer

L2 regularization

Answer

Dropout

Answer

Batch normalization

Question 3

What is the primary effect of dropout regularization during training?

Accepted Answer

It randomly deactivates a fraction of neurons, preventing co-adaptation

Answer

It normalizes the activations at each layer

Answer

It reduces the learning rate over time

Answer

It clips gradient values to prevent explosion

Question 4

What is the bias-variance tradeoff in the context of neural networks?

Accepted Answer

Balancing underfitting (high bias) against overfitting (high variance)

Answer

Balancing the number of layers against the number of neurons per layer

Answer

Balancing training speed against model accuracy

Answer

Balancing weight initialization against learning rate

Question 5

Which of the following is a symptom of underfitting in a neural network?

Accepted Answer

Low training accuracy and low test accuracy

Answer

High training accuracy and low test accuracy

Answer

High training accuracy and high test accuracy

Answer

Low training accuracy and high test accuracy

Question 6

What does L2 regularization (weight decay) do to the weights during training?

Accepted Answer

It encourages weights to remain small by penalizing their squared magnitude

Answer

It sets small weights to exactly zero

Answer

It increases large weights to improve learning speed

Answer

It normalizes weights to have unit norm after each update

Question 7

Early stopping is a regularization technique that halts training when:

Accepted Answer

Validation loss stops improving or begins to increase

Answer

Training loss reaches zero

Answer

The learning rate drops below a threshold

Answer

All weight values converge to the same value