CNN - Convolutional Neural Networks Overfitting and Regularization Questions and Answers

Question 1

A CNN model achieves 99% accuracy on the training dataset but only 75% on the validation dataset. Which phenomenon is occurring, and what is a common technique to address it?

Accepted Answer

Overfitting; apply Dropout to the fully connected layers.

Answer

The significant performance gap between training and validation accuracy is a classic sign of overfitting, where the model has learned the training data too well, including its noise, and fails to generalize to new data. [24, 27] Dropout is a widely used regularization technique that randomly sets a fraction of neuron activations to zero during training, which helps prevent complex co-adaptations and improves generalization. [3, 18, 20]

Question 2

What is the primary effect of L1 regularization on the weights of a CNN?

Accepted Answer

It encourages some weights to become exactly zero, leading to a sparse model.

Answer

L1 regularization adds a penalty to the loss function proportional to the absolute value of the weights. [12] This has the effect of pushing the weights of less important features towards exactly zero, a process that results in a 'sparse' model where many weights are zero. [8, 9, 19] This can also be seen as a form of automatic feature selection.

Question 3

A data scientist is training a CNN on a small, specialized dataset of industrial parts. To prevent the model from memorizing the limited training examples, they decide to artificially expand the dataset. Which of the following techniques are they employing?

Accepted Answer

Data Augmentation

Answer

Data augmentation is the process of artificially increasing the size of a training dataset by creating modified versions of the existing data. [15, 17] For images, this includes techniques like rotation, flipping, scaling, and color shifting. [4, 12, 18] This helps the model generalize better by exposing it to a wider variety of training examples, which is a powerful method for combating overfitting, especially with small datasets. [1, 15]

Question 4

Which of the following regularization techniques involves halting the training process when the model's performance on a validation set stops improving?

Accepted Answer

Early Stopping

Answer

Early stopping is a form of regularization where the model's performance on a separate validation dataset is monitored during training. If the validation performance (e.g., accuracy or loss) does not improve for a specified number of consecutive epochs (the 'patience'), the training process is halted to prevent the model from overfitting to the training data. [2, 5, 10, 16]

Question 5

What is the primary goal of using regularization techniques like Dropout or L2 regularization when training a Convolutional Neural Network?

Accepted Answer

To reduce the model's complexity and improve its ability to generalize to unseen data.

Answer

The fundamental purpose of regularization is to prevent overfitting. [2, 12] Overfitting occurs when a model learns the training data too well and fails to generalize to new, unseen data. [24, 27] Regularization techniques introduce constraints or penalties on the model's parameters (like L2 regularization) or its structure during training (like Dropout) to reduce its complexity and force it to learn more robust, generalizable patterns. [6, 11]

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Overfitting and Regularization Questions and Answers