Neural Network Regularization and Overfitting

Question 1

What does overfitting in a neural network mean?

Accepted Answer

The model performs well on training data but poorly on unseen data

Answer

Overfitting occurs when a model learns the training data too well, including noise and irrelevant patterns, resulting in poor generalization to new data.

Question 2

Which regularization technique adds the sum of absolute values of weights to the loss function?

Accepted Answer

L1 regularization

Answer

L1 regularization (Lasso) penalizes the loss function by adding the sum of the absolute values of the weights, promoting sparsity in the model.

Question 3

What is the primary effect of dropout regularization during training?

Accepted Answer

It randomly deactivates a fraction of neurons, preventing co-adaptation

Answer

Dropout randomly sets a fraction of neuron activations to zero during each training step, forcing the network to learn redundant representations and reducing co-adaptation.

Question 4

What is the bias-variance tradeoff in the context of neural networks?

Accepted Answer

Balancing underfitting (high bias) against overfitting (high variance)

Answer

The bias-variance tradeoff describes the tension between a model that is too simple (high bias, underfits) and one that is too complex (high variance, overfits).

Question 5

Which of the following is a symptom of underfitting in a neural network?

Accepted Answer

Low training accuracy and low test accuracy

Answer

Underfitting occurs when the model is too simple to capture the underlying data patterns, resulting in poor performance on both the training set and the test set.

Neural Network Practice Test

Neural Network Practice Test

Neural Network Regularization and Overfitting