Deep Learning Transfer Learning and Fine-Tuning 2

Question 1

What is 'domain adaptation' in transfer learning?

Accepted Answer

Adapting a model trained in one data distribution to work well on a different but related distribution

Answer

Changing the architecture of a neural network to fit a new problem

Answer

Collecting more training data from the target domain

Answer

Converting a regression model into a classification model

Question 2

In NLP transfer learning, what does BERT's pre-training primarily involve?

Accepted Answer

Masked language modeling and next sentence prediction

Answer

Next sentence classification only

Answer

Autoregressive language generation

Answer

Supervised text classification on Wikipedia

Question 3

What is 'catastrophic forgetting' in neural networks?

Accepted Answer

The tendency of a neural network to abruptly lose previously learned information when trained on new data

Answer

A model failing to converge due to exploding gradients

Answer

GPU memory overflow during large batch training

Answer

Gradient vanishing in very deep networks

Question 4

What is 'zero-shot learning' in deep learning?

Accepted Answer

A model's ability to recognize or solve tasks it has never explicitly seen during training

Answer

Training a model without any data augmentation

Answer

Skipping the validation step during model training

Answer

Training with zero regularization

Question 5

In transfer learning, what is the primary purpose of replacing the original output head with a new one?

Accepted Answer

To adapt the model's final predictions to the number and type of classes in the new target task

Answer

To increase the depth of the network

Answer

To reduce memory consumption during inference

Answer

To improve gradient flow during backpropagation

Question 6

Which of the following best describes cross-domain transfer learning?

Accepted Answer

Applying knowledge learned in one modality or domain (e.g., images) to a different domain (e.g., medical scans) with a distribution gap

Answer

Applying a model trained on images to another image dataset with the same classes

Answer

Transferring model weights between different hardware platforms

Answer

Using the same dataset for both pre-training and fine-tuning

Question 7

What is 'discriminative fine-tuning' as used in language model transfer?

Accepted Answer

Applying different learning rates to different layers, with lower rates for earlier layers

Answer

Fine-tuning only the discriminator in a GAN

Answer

Training only on the hardest examples in the dataset

Answer

Using a discriminative loss instead of cross-entropy