Deep Learning Transfer Learning and Fine-Tuning 2

Question 1

What is 'domain adaptation' in transfer learning?

Accepted Answer

Adapting a model trained in one data distribution to work well on a different but related distribution

Answer

Domain adaptation addresses the distribution shift between source and target domains so that a model trained on the source performs well on the target without requiring extensive target labels.

Question 2

In NLP transfer learning, what does BERT's pre-training primarily involve?

Accepted Answer

Masked language modeling and next sentence prediction

Answer

BERT is pre-trained using Masked Language Modeling (predicting randomly masked tokens) and Next Sentence Prediction, enabling it to learn deep bidirectional representations.

Question 3

What is 'catastrophic forgetting' in neural networks?

Accepted Answer

The tendency of a neural network to abruptly lose previously learned information when trained on new data

Answer

Catastrophic forgetting occurs when training a network on a new task overwrites the weights that encoded knowledge from previous tasks, causing performance on those tasks to degrade sharply.

Question 4

What is 'zero-shot learning' in deep learning?

Accepted Answer

A model's ability to recognize or solve tasks it has never explicitly seen during training

Answer

Zero-shot learning enables a model to generalize to unseen classes or tasks at inference time by leveraging semantic descriptions, attribute vectors, or language embeddings learned during training.

Question 5

In transfer learning, what is the primary purpose of replacing the original output head with a new one?

Accepted Answer

To adapt the model's final predictions to the number and type of classes in the new target task

Answer

The original head is task-specific to the source dataset; replacing it with a new head sized to the target task's output space allows the pre-trained backbone's features to be applied to a different problem.

Deep Learning Practice Test

Deep Learning Practice Test

Deep Learning Transfer Learning and Fine-Tuning 2