CNN - Convolutional Neural Networks Transfer Learning with CNNs Questions and Answers

Question 1

A developer is building a flower species classifier using a small, custom dataset of approximately 900 images. They decide to use a ResNet50 model pre-trained on ImageNet. To leverage the pre-trained features effectively while minimizing the risk of overfitting, which transfer learning strategy should they implement first?

Accepted Answer

Freeze all convolutional layers and train only a new, randomly initialized classifier head.

Answer

With a small dataset, there is a high risk of overfitting if the entire network is trained. The best initial strategy is feature extraction, which involves freezing the pre-trained convolutional layers to use their powerful, generic feature detection capabilities and only training the new classifier head on the small dataset.

Question 2

What is the primary motivation for 'freezing' the early convolutional layers of a pre-trained CNN when applying transfer learning to a new task?

Accepted Answer

To preserve the generic, low-level feature detectors (e.g., edges, colors, textures) that the model learned from the original large-scale dataset.

Answer

The initial layers of a CNN learn to detect general features like edges, corners, and color blobs, which are applicable to most computer vision tasks. Freezing these layers prevents their weights from being updated, thereby preserving this fundamental knowledge and preventing it from being corrupted by training on a potentially small or different dataset.

Question 3

Which of the following best describes the strategy of 'fine-tuning' in the context of transfer learning with a pre-trained CNN?

Accepted Answer

Replacing the final classification layer and subsequently unfreezing some of the later convolutional layers to continue training them on the new data, typically with a very low learning rate.

Answer

Fine-tuning involves not only replacing the classifier head but also unfreezing some of the deeper, more specialized layers of the pre-trained model. These layers are then trained on the new dataset with a low learning rate to subtly adjust their weights to the new task without drastically altering the learned features.

Question 4

A team is adapting a powerful ImageNet pre-trained model for a new task of classifying industrial machine parts from a large, proprietary dataset of over 200,000 images. The visual characteristics of the machine parts are significantly different from the natural images in ImageNet. What is the most effective transfer learning approach?

Accepted Answer

Unfreeze most or all of the layers and fine-tune the entire network using the pre-trained weights as a starting point.

Answer

Because the new dataset is large, there is less risk of overfitting. Since the data domain is different, the model needs to learn more specialized features. The optimal approach is to use the pre-trained weights as a superior alternative to random initialization and fine-tune the entire network, allowing it to adapt its learned features to the new domain of machine parts.

Question 5

When fine-tuning a pre-trained CNN, it is a common best practice to use a significantly smaller learning rate for the convolutional layers than for the newly added classifier head. What is the primary reason for this practice, often called 'differential learning rates'?

Accepted Answer

To make only small, careful adjustments to the pre-trained weights, preventing the catastrophic forgetting of valuable learned features.

Answer

The weights of the pre-trained convolutional layers already contain a vast amount of useful information. A high learning rate would cause large updates, potentially destroying this information. Using a much smaller learning rate ensures that these weights are only slightly adjusted (fine-tuned) to become more relevant to the new task, preserving the core of their learned knowledge.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Transfer Learning with CNNs Questions and Answers