Deep Learning Transfer Learning and Fine-Tuning 1 — Questions and Answers
Question 1: What is transfer learning in deep learning?
- Training a model from scratch on a new dataset
- Reusing knowledge from a model trained on one task to improve performance on a different task (Correct answer)
- Transferring data between different storage systems
- Using multiple GPUs to speed up training
Correct answer: Reusing knowledge from a model trained on one task to improve performance on a different task
Transfer learning leverages knowledge (weights and representations) acquired from a source task to improve learning efficiency and performance on a related target task.
Question 2: Which of the following is a pre-trained model commonly used for transfer learning in computer vision?
- LSTM
- Word2Vec
- ResNet (Correct answer)
- ARIMA
Correct answer: ResNet
ResNet (Residual Network) is a widely used pre-trained convolutional neural network originally trained on ImageNet, making it a standard backbone for computer vision transfer learning.
Question 3: What does 'fine-tuning' mean in the context of transfer learning?
- Adjusting hyperparameters such as batch size and epoch count
- Continuing to train some or all layers of a pre-trained model on a new dataset (Correct answer)
- Pruning unused neurons from the network
- Selecting which features to include in the input
Correct answer: Continuing to train some or all layers of a pre-trained model on a new dataset
Fine-tuning involves taking a pre-trained model and continuing its training on a new (often smaller) dataset so it adapts to the target task while retaining prior knowledge.
Question 4: Which layers are typically frozen during the initial stages of transfer learning?
- The final classification layer
- The early convolutional layers that capture low-level features (Correct answer)
- The batch normalization layers
- All layers including the output layer
Correct answer: The early convolutional layers that capture low-level features
Early layers learn general low-level features (edges, textures) that are broadly useful, so they are frozen to preserve this generic knowledge while later task-specific layers are updated.
Question 5: What is a 'feature extractor' in the context of transfer learning?
- A preprocessing step that normalizes input data
- Using a pre-trained network's intermediate representations as input to a new model without updating the pre-trained weights (Correct answer)
- A method for selecting the most informative training examples
- A technique for compressing model weights
Correct answer: Using a pre-trained network's intermediate representations as input to a new model without updating the pre-trained weights
When using a pre-trained model as a feature extractor, its weights are frozen and its intermediate activations serve as fixed feature representations fed into a newly trained classifier.
Question 6: What is the primary advantage of transfer learning over training a model from scratch?
- It always produces higher accuracy regardless of dataset size
- It eliminates the need for labeled data entirely
- It reduces the amount of labeled data and compute required to achieve good performance (Correct answer)
- It prevents overfitting in all scenarios
Correct answer: It reduces the amount of labeled data and compute required to achieve good performance
Transfer learning enables models to achieve strong performance with far fewer labeled examples and less training time by starting from representations already learned on large datasets.
Question 7: Which strategy is generally recommended when fine-tuning a pre-trained model?
- Use a high learning rate to quickly adapt all layers
- Use a lower learning rate than was used in original pre-training to avoid destroying learned features (Correct answer)
- Randomly reinitialize all weights before fine-tuning
- Only train the input embedding layers
Correct answer: Use a lower learning rate than was used in original pre-training to avoid destroying learned features
A small learning rate during fine-tuning makes incremental adjustments to pre-trained weights, preserving valuable learned representations while adapting to the new task.
Deep Learning Transfer Learning and Fine-Tuning 1
What is transfer learning in deep learning?