Deep Learning (Data Scientist) 5

Question 1

A data scientist fine-tunes a large language model on task B and finds it now performs poorly on the original task A. What is this called?

Accepted Answer

Catastrophic forgetting

Answer

Catastrophic forgetting occurs when new-task training overwrites weights encoding previously learned knowledge.

Question 2

Which initialization scheme is specifically designed for layers with ReLU activations?

Accepted Answer

He (Kaiming) initialization

Answer

He initialization scales variance by 2/n to compensate for ReLU zeroing out half of its inputs on average.

Question 3

Class imbalance is severe: 99% negative, 1% positive. Which combination best supports training a useful classifier?

Accepted Answer

Class-weighted loss or resampling, evaluated with precision-recall metrics

Answer

Weighting or resampling counteracts the imbalance, and precision-recall metrics reveal minority-class performance that accuracy hides.

Question 4

What is the primary benefit of mixed-precision (FP16/FP32) training on modern GPUs?

Accepted Answer

Faster computation and lower memory use with minimal accuracy loss

Answer

Half-precision arithmetic exploits tensor cores and halves memory traffic while loss scaling preserves numerical stability.

Question 5

In self-supervised learning, how does contrastive pretraining like SimCLR create labels?

Accepted Answer

It treats augmented views of the same image as positives and other images as negatives

Answer

Contrastive learning generates supervision by pulling together embeddings of two augmentations of one image and pushing apart embeddings of different images.

Deep Learning Practice Test

Deep Learning Practice Test

Deep Learning (Data Scientist) 5