Deep Learning Transfer Learning and Fine-Tuning 3

Question 1

What is 'few-shot learning' in deep learning?

Accepted Answer

A learning paradigm where a model generalizes to new tasks from only a handful of labeled examples

Answer

Few-shot learning focuses on enabling models to learn new concepts from very few (e.g., 1–5) labeled examples per class, leveraging prior knowledge to make this generalization possible.

Question 2

Which approach is the core idea behind Model-Agnostic Meta-Learning (MAML)?

Accepted Answer

Learning an initialization of model parameters that can be quickly adapted to new tasks with few gradient steps

Answer

MAML meta-trains model parameters to lie in a region of the loss landscape from which a small number of gradient updates on a new task leads to strong performance, enabling fast adaptation.

Question 3

What is 'knowledge distillation' in deep learning?

Accepted Answer

Training a smaller student model to mimic the output distribution of a larger teacher model

Answer

Knowledge distillation trains a compact student model to match the soft probability outputs (dark knowledge) of a larger, more capable teacher model, transferring learned knowledge without the teacher's computational cost.

Question 4

What is 'domain shift' and why does it challenge transfer learning?

Accepted Answer

A statistical difference between the source and target data distributions that causes a drop in model performance

Answer

Domain shift refers to the mismatch between the statistical properties of source training data and target deployment data, which causes models to underperform because their learned features may not generalize across the gap.

Question 5

What is 'gradual unfreezing' in the context of fine-tuning?

Accepted Answer

Progressively unfreezing layers from the top (task-specific) toward the bottom (general) as training proceeds

Answer

Gradual unfreezing starts by training only the final layers, then incrementally unfreezes earlier layers epoch by epoch, allowing lower-level representations to adapt without destabilizing them immediately.

Deep Learning Practice Test

Deep Learning Practice Test

Deep Learning Transfer Learning and Fine-Tuning 3