Deep Learning Transfer Learning and Fine-Tuning 3

Question 1

What is 'few-shot learning' in deep learning?

Accepted Answer

A learning paradigm where a model generalizes to new tasks from only a handful of labeled examples

Answer

Training a model with very few epochs

Answer

Using a small neural network with few parameters

Answer

A regularization technique that drops most neurons

Question 2

Which approach is the core idea behind Model-Agnostic Meta-Learning (MAML)?

Accepted Answer

Learning an initialization of model parameters that can be quickly adapted to new tasks with few gradient steps

Answer

Pre-training a model on a fixed large dataset and never updating it

Answer

Using an ensemble of pre-trained models without fine-tuning

Answer

Replacing gradient descent with evolutionary algorithms

Question 3

What is 'knowledge distillation' in deep learning?

Accepted Answer

Training a smaller student model to mimic the output distribution of a larger teacher model

Answer

Extracting symbolic rules from a neural network for interpretability

Answer

Compressing weights using quantization techniques

Answer

Removing redundant neurons via pruning

Question 4

What is 'domain shift' and why does it challenge transfer learning?

Accepted Answer

A statistical difference between the source and target data distributions that causes a drop in model performance

Answer

A change in the model architecture between pre-training and fine-tuning

Answer

Switching from one optimization algorithm to another

Answer

Changing the number of output classes between tasks

Question 5

What is 'gradual unfreezing' in the context of fine-tuning?

Accepted Answer

Progressively unfreezing layers from the top (task-specific) toward the bottom (general) as training proceeds

Answer

Slowly increasing the batch size during training

Answer

Annealing the learning rate to zero over many epochs

Answer

Gradually adding more layers to the network during training

Question 6

In NLP transfer learning, what does the term 'downstream task' refer to?

Accepted Answer

The specific application task (e.g., sentiment analysis, NER) that a pre-trained language model is fine-tuned or adapted to solve

Answer

A task performed on data that flows downstream in a data pipeline

Answer

The final layer of a transformer model

Answer

A lower-priority training objective used during pre-training

Question 7

Which of the following transfer learning strategies is most appropriate when the target dataset is small and similar to the source dataset?

Accepted Answer

Use the pre-trained model as a fixed feature extractor without fine-tuning the convolutional base

Answer

Train the entire network end-to-end from scratch

Answer

Fine-tune all layers with a high learning rate

Answer

Replace the entire architecture with a simpler model