Deep Learning Fundamentals Assessment — Questions and Answers

Question 1: What is cross-attention in a Transformer decoder?

Attention where queries come from the decoder and keys/values come from the encoder, allowing decoder to attend to input context (Correct answer)
Attention between adjacent layers in the encoder
Shared attention weights between encoder and decoder
Attention applied across multiple training examples in a batch

Correct answer: Attention where queries come from the decoder and keys/values come from the encoder, allowing decoder to attend to input context

Cross-attention lets each decoder position query the encoder's output representations, allowing the decoder to selectively focus on relevant parts of the input sequence.

Question 2: What is the primary advantage of transfer learning over training a model from scratch?

It reduces the amount of labeled data and compute required to achieve good performance (Correct answer)
It prevents overfitting in all scenarios
It eliminates the need for labeled data entirely
It always produces higher accuracy regardless of dataset size

Correct answer: It reduces the amount of labeled data and compute required to achieve good performance

Transfer learning enables models to achieve strong performance with far fewer labeled examples and less training time by starting from representations already learned on large datasets.

Question 3: In a supervised learning task, the number of neurons in the output layer should match the number of classes (where the number of classes is larger than 2). Is this statement true or false?

A) False (Correct answer)
B) True

Correct answer: A) False

This statement is false. While for multi-class classification (more than two classes) using a softmax activation, the number of output neurons typically matches the number of classes, this is not universally true. For binary classification (two classes), a single output neuron with a sigmoid activation function is often sufficient. This neuron outputs the probability of one class, with the other class's probability being 1 minus that value.

Question 4: When the data is too large to handle in RAM at the same time, which gradient technique is more advantageous?

B) Full Batch Gradient Descent
A) Stochastic Gradient Descent (Correct answer)

Correct answer: A) Stochastic Gradient Descent

Stochastic Gradient Descent (SGD) is more advantageous when dealing with datasets too large to fit into RAM. Unlike Full Batch Gradient Descent, which computes gradients over the entire dataset, SGD updates model weights after processing each individual training example or a small mini-batch. This approach significantly reduces memory requirements, as it doesn't need to load the entire dataset into memory for each gradient calculation, making it scalable for large-scale data.

Question 5: Which loss function is most commonly used for multi-class classification problems?

Categorical Cross-Entropy (Correct answer)
Hinge Loss
Binary Cross-Entropy
Mean Squared Error

Correct answer: Categorical Cross-Entropy

Categorical Cross-Entropy measures the dissimilarity between predicted probability distributions and one-hot encoded true class labels.

Question 6: Which loss function is most appropriate for a multi-class classification network with a softmax output layer?

Categorical cross-entropy (Correct answer)
Mean squared error
Huber loss
Hinge loss

Correct answer: Categorical cross-entropy

Categorical cross-entropy directly measures the divergence between the softmax probability distribution and the one-hot target.

Question 7: What are the two main components of a Generative Adversarial Network (GAN)?

Policy network and value network
Encoder and decoder
Generator and discriminator trained in opposition to each other (Correct answer)
Inference network and sampling network

Correct answer: Generator and discriminator trained in opposition to each other

A GAN consists of a generator that creates fake samples and a discriminator that distinguishes real from fake, with each trained adversarially to improve the other.

Question 8: Which technique randomly sets neuron outputs to zero during training to reduce overfitting?

L2 regularization
Dropout (Correct answer)
Batch normalization
Weight decay

Correct answer: Dropout

Dropout randomly deactivates a fraction of neurons during each training step, preventing co-adaptation and acting as an ensemble of sub-networks.

Question 9: What problem does the forget gate in an LSTM address?

Exploding activation values
Normalizing the input to the cell
Deciding how much of the previous cell state to retain or discard (Correct answer)
Mapping hidden states to output predictions

Correct answer: Deciding how much of the previous cell state to retain or discard

The forget gate outputs values between 0 and 1 for each cell state element, controlling how much prior context is carried forward and what is discarded.

Question 10: What is the key property of flow-based generative models such as RealNVP?

They encode inputs to discrete latent codes
They learn invertible transformations allowing exact likelihood computation and efficient sampling in both directions (Correct answer)
They model the data distribution using a recurrent prior
They use adversarial training between two networks

Correct answer: They learn invertible transformations allowing exact likelihood computation and efficient sampling in both directions

Flow-based models use a series of invertible (bijective) transformations with tractable Jacobians, enabling exact log-likelihood evaluation and straightforward sampling by inverting the flow.

Question 11: What is the purpose of warm-up in learning rate scheduling for Transformer training?

Gradually increasing the learning rate from near zero during the first N steps before decaying (Correct answer)
Warming up the GPU before intensive training begins
Starting with a high learning rate to converge quickly
Using a fixed learning rate for the first epoch

Correct answer: Gradually increasing the learning rate from near zero during the first N steps before decaying

Warm-up gradually increases the learning rate from zero, stabilizing early training when model parameters and optimizer state estimates are poorly initialized.

Question 12: What is the main limitation of vanilla Seq2Seq models for long input sequences?

They cannot be trained with gradient-based methods
The fixed-size context vector loses information when compressing long sequences (Correct answer)
They cannot handle variable-length inputs
They require labeled data for every time step

Correct answer: The fixed-size context vector loses information when compressing long sequences

A single fixed-length context vector must encode all information from an arbitrarily long input, causing information loss for long sequences.

Question 13: The human brain is thought to have inspired a neural network model. The neural network is made up of many different components. Each neuron receives an input, processes it, and then outputs. Which of the following statements represents a genuine neuron correctly?

A neuron has only one input and only one output.
All of the following statements are correct. (Correct answer)
Although a neuron has only one input, it has several outputs.
Multiple inputs and outputs are found in a neuron.
A neuron has several inputs but only one output.

Correct answer: All of the following statements are correct.

The human brain contains a vast diversity of neuron types, each with unique structures and functions. While a typical neuron receives multiple inputs (dendrites) and its single axon can branch to provide multiple outputs to other neurons, some specialized neurons or simplified models might emphasize different aspects of connectivity. Therefore, depending on the specific neuron type or the level of abstraction, various statements about input/output configurations could be considered correct in a broad context.

Question 14: In transfer learning, what is the primary purpose of replacing the original output head with a new one?

To increase the depth of the network
To reduce memory consumption during inference
To improve gradient flow during backpropagation
To adapt the model's final predictions to the number and type of classes in the new target task (Correct answer)

Correct answer: To adapt the model's final predictions to the number and type of classes in the new target task

The original head is task-specific to the source dataset; replacing it with a new head sized to the target task's output space allows the pre-trained backbone's features to be applied to a different problem.

Question 15: What loss function does a standard GAN generator minimize?

Reconstruction loss between generated and real images
Cross-entropy between generated distribution and target distribution
The negative log-probability of the discriminator classifying generated samples as real (Correct answer)
Mean squared error between latent codes and output images

Correct answer: The negative log-probability of the discriminator classifying generated samples as real

The generator is trained to maximize the discriminator's probability of labeling generated samples as real, equivalent to minimizing the negative log of that probability.

Question 16: What is data augmentation used for when training CNNs?

Speeding up gradient computation
Increasing inference speed
Reducing the number of model parameters
Artificially expanding training data with transformations to improve generalization (Correct answer)

Correct answer: Artificially expanding training data with transformations to improve generalization

Data augmentation applies random transforms like flips, crops, and color jitter to training images, exposing the model to more variation and reducing overfitting.

Question 17: What problem does batch normalization primarily address?

Overfitting on training data
Class imbalance in datasets
Internal covariate shift causing unstable training (Correct answer)
Exploding gradient values

Correct answer: Internal covariate shift causing unstable training

Batch normalization reduces internal covariate shift by normalizing layer inputs to have zero mean and unit variance, stabilizing and accelerating training.

Question 18: When training a deep network, exploding gradients cause the loss to become NaN. Which is the most direct remedy?

Increase the number of epochs
Disable weight initialization
Apply gradient clipping (Correct answer)
Switch from ReLU to sigmoid activations everywhere

Correct answer: Apply gradient clipping

Gradient clipping caps the gradient norm, preventing runaway updates that produce numerical overflow.

Question 19: What is the exploding gradient problem in RNNs?

Activations becoming negative during training
Gradients growing exponentially through many time steps, causing unstable large weight updates (Correct answer)
Loss increasing to infinity due to wrong architecture
Too many neurons being activated simultaneously

Correct answer: Gradients growing exponentially through many time steps, causing unstable large weight updates

When gradients are multiplied across many time steps, they can grow exponentially, producing extremely large weight updates that destabilize or diverge training.

Question 20: What is text-to-image generation and which model type is currently dominant for this task?

Transcribing handwritten text in images; CNNs
Converting image captions to text descriptions; RNNs
Generating photorealistic images from natural language descriptions; diffusion models conditioned on text embeddings (Correct answer)
Retrieving images from a database matching a text query; retrieval models

Correct answer: Generating photorealistic images from natural language descriptions; diffusion models conditioned on text embeddings

Text-to-image generation produces images matching a text prompt; diffusion models conditioned on CLIP or text encoder embeddings are the current state-of-the-art approach.

Question 21: In sequence-to-sequence (Seq2Seq) models, what is the role of the encoder?

Computing cross-entropy loss over predictions
Compressing the input sequence into a fixed-length context vector (Correct answer)
Generating the output sequence token by token
Applying attention over the input

Correct answer: Compressing the input sequence into a fixed-length context vector

The encoder processes the input sequence and compresses it into a context vector (the final hidden state) that summarizes the input for the decoder.

Question 22: What is the computational complexity of self-attention with respect to sequence length n?

O(n)
O(n log n)
O(n³)
O(n²) (Correct answer)

Correct answer: O(n²)

Self-attention computes pairwise interactions between all n tokens, requiring O(n²) time and memory, which becomes prohibitive for very long sequences.

Question 23: What is backpropagation through time (BPTT)?

Evaluating the model on future data
Applying dropout at each time step
Unrolling an RNN across time steps and applying backpropagation to compute gradients (Correct answer)
Using time-series data to initialize RNN weights

Correct answer: Unrolling an RNN across time steps and applying backpropagation to compute gradients

BPTT unfolds the RNN computation graph across all time steps and applies the standard backpropagation algorithm to compute gradients with respect to all shared weights.

Question 24: What is 'gradual unfreezing' in the context of fine-tuning?

Slowly increasing the batch size during training
Annealing the learning rate to zero over many epochs
Gradually adding more layers to the network during training
Progressively unfreezing layers from the top (task-specific) toward the bottom (general) as training proceeds (Correct answer)

Correct answer: Progressively unfreezing layers from the top (task-specific) toward the bottom (general) as training proceeds

Gradual unfreezing starts by training only the final layers, then incrementally unfreezes earlier layers epoch by epoch, allowing lower-level representations to adapt without destabilizing them immediately.

Question 25: Translation invariance is kept when a pooling layer is added to a convolutional neural network. Is this statement true or false?

B) True (Correct answer)
A) False

Correct answer: B) True

This statement is true. Pooling layers, such as max pooling or average pooling, are crucial components in Convolutional Neural Networks that contribute to translation invariance. By downsampling the feature maps, pooling makes the network less sensitive to the exact position of a feature within the input. If a feature shifts slightly, the pooling operation can still detect it in the same pooled region, enhancing the network's ability to recognize objects regardless of minor translations.

Question 26: What is a Variational Autoencoder (VAE) and how does it differ from a standard autoencoder?

A VAE is identical to a standard autoencoder but uses convolutional layers
A VAE has no decoder, only an encoder
A VAE encodes inputs as probability distributions in latent space rather than fixed vectors, enabling principled sampling and generation (Correct answer)
A VAE uses adversarial training instead of reconstruction loss

Correct answer: A VAE encodes inputs as probability distributions in latent space rather than fixed vectors, enabling principled sampling and generation

A VAE learns a probabilistic encoder mapping inputs to distributions (mean + variance) and uses the reparameterization trick to enable backpropagation through the sampling step.

Question 27: What is the universal approximation theorem?

Any network can approximate any function with infinite layers
Neural networks converge to global optima given enough data
A single hidden layer with enough neurons can approximate any continuous function (Correct answer)
Deep networks always outperform shallow ones

Correct answer: A single hidden layer with enough neurons can approximate any continuous function

The universal approximation theorem states that a feedforward network with one hidden layer and sufficient neurons can approximate any continuous function on a compact subset.

Question 28: What does a convolutional filter (kernel) do when applied to an input feature map?

Computes a dot product between the filter and a local patch of the input (Correct answer)
Transposes the input matrix
Averages all pixel values globally
Performs element-wise addition

Correct answer: Computes a dot product between the filter and a local patch of the input

A convolution filter slides over the input and computes the dot product between its weights and each local receptive field, producing an activation map.

Question 29: What does an object detection CNN like YOLO do differently from a standard image classification CNN?

It only processes images in grayscale
It uses recurrent layers instead of convolutional layers
It predicts both class labels and bounding box coordinates simultaneously (Correct answer)
It does not use any convolutional layers

Correct answer: It predicts both class labels and bounding box coordinates simultaneously

Object detection networks like YOLO predict class probabilities and bounding box coordinates in a single forward pass, enabling real-time detection of multiple objects.

Question 30: What is conditional generation in generative models?

Applying post-processing conditions to raw generated outputs
Training the model only on samples satisfying a quality threshold
Generating samples only when the model has high confidence
Conditioning the generative process on a label or other input to control the attributes of generated samples (Correct answer)

Correct answer: Conditioning the generative process on a label or other input to control the attributes of generated samples

Conditional generation feeds class labels, text prompts, or other conditioning signals to the generative model, enabling control over the category or attributes of generated outputs.

Question 31: What is the Wasserstein GAN (WGAN) designed to address?

Slow convergence in the discriminator
Class-conditional generation without labels
Training instability and mode collapse by using the Wasserstein distance as a more stable loss (Correct answer)
Mode collapse in image synthesis

Correct answer: Training instability and mode collapse by using the Wasserstein distance as a more stable loss

WGAN replaces the standard GAN loss with the Wasserstein distance (Earth Mover's distance), providing more meaningful gradients even when generator and real distributions don't overlap.

Question 32: In a feedforward neural network, what direction does information flow during inference?

Bidirectionally through all layers
Backward through layers
Forward from input to output (Correct answer)
Only through skip connections

Correct answer: Forward from input to output

During inference, data passes forward from the input layer through hidden layers to the output layer without any backward pass.

Question 33: What is hyperparameter tuning and which approach searches the space most efficiently?

Tuning only the learning rate; grid search suffices
Manually adjusting weights; manual search is most efficient
Adjusting model architecture during training; random search is always best
Searching for optimal training configuration values; Bayesian optimization generally outperforms random or grid search (Correct answer)

Correct answer: Searching for optimal training configuration values; Bayesian optimization generally outperforms random or grid search

Hyperparameter tuning searches for the best learning rate, batch size, architecture choices, etc.; Bayesian optimization uses a probabilistic surrogate model to guide the search efficiently.

Question 34: What does 'stateful' mode mean for an RNN in frameworks like Keras?

The model saves weights after every batch
Gradients are accumulated across multiple forward passes
The hidden state from the end of one batch is used as the initial state of the next batch (Correct answer)
The model stores training data in memory

Correct answer: The hidden state from the end of one batch is used as the initial state of the next batch

In stateful mode, the final hidden state of each batch becomes the initial hidden state for the subsequent batch, enabling processing of very long sequences across batches.

Question 35: A data scientist notices training loss decreasing while validation loss starts rising after epoch 15. What is the most appropriate response?

Increase the learning rate
Remove the validation set
Apply early stopping or regularization (Correct answer)
Add more hidden layers

Correct answer: Apply early stopping or regularization

Diverging training and validation loss indicates overfitting, which early stopping or regularization directly addresses.

Question 36: What does 'stride' refer to in a convolutional layer?

The padding added to input borders
The number of output channels
The number of steps the filter moves per application (Correct answer)
The size of the convolutional filter

Correct answer: The number of steps the filter moves per application

Stride controls how many pixels the filter shifts between consecutive applications, with larger strides producing smaller output feature maps.

Question 37: Which activation function suffers from the 'dying ReLU' problem?

ReLU (Correct answer)
Sigmoid
Softmax
Tanh

Correct answer: ReLU

ReLU neurons can permanently output zero for all inputs when weights push the input to always be negative, rendering them inactive.

Question 38: What are the steps involved in employing a gradient descent algorithm? 1. Reiterate until you find the best weights of network 2. Go to each neurons which contributes to the error and change its respective values to reduce the error 3. Calculate error between the actual value and the predicted value 4.Pass an input through the network and get values from output layer 5. Initialize random weight and bias

4, 3, 1, 5, 2
1, 2, 3, 4, 5
5, 4, 3, 2, 1 (Correct answer)
3, 2, 1, 5, 4

Correct answer: 5, 4, 3, 2, 1

The gradient descent algorithm starts by initializing random weights and biases (5). An input is then passed through the network to generate a prediction (4), and the error between this prediction and the actual value is calculated (3). Based on this error, the algorithm determines how to adjust the weights and biases of each neuron to reduce the error (2). This entire sequence is repeated iteratively until the network's weights are optimized and the error is minimized (1).

Question 39: What is a many-to-one RNN architecture used for?

Mapping an input sequence to a single output such as sentiment classification (Correct answer)
Processing a single input to produce a sequence
Generating a sequence from a single input
Mapping sequences of equal length

Correct answer: Mapping an input sequence to a single output such as sentiment classification

Many-to-one RNNs read an entire input sequence and produce a single output, commonly used for tasks like sentence sentiment classification or document categorization.

Question 40: What distinguishes a recurrent neural network (RNN) from a feedforward network?

RNNs have no activation functions
RNNs use convolutional operations
RNNs only process fixed-size inputs
RNNs have feedback connections that carry hidden state across time steps (Correct answer)

Correct answer: RNNs have feedback connections that carry hidden state across time steps

RNNs pass a hidden state from one time step to the next, allowing the network to maintain memory of previous inputs in a sequence.

Question 41: Which layers are typically frozen during the initial stages of transfer learning?

The batch normalization layers
All layers including the output layer
The early convolutional layers that capture low-level features (Correct answer)
The final classification layer

Correct answer: The early convolutional layers that capture low-level features

Early layers learn general low-level features (edges, textures) that are broadly useful, so they are frozen to preserve this generic knowledge while later task-specific layers are updated.

Question 42: What is a 'feature extractor' in the context of transfer learning?

Using a pre-trained network's intermediate representations as input to a new model without updating the pre-trained weights (Correct answer)
A preprocessing step that normalizes input data
A technique for compressing model weights
A method for selecting the most informative training examples

Correct answer: Using a pre-trained network's intermediate representations as input to a new model without updating the pre-trained weights

When using a pre-trained model as a feature extractor, its weights are frozen and its intermediate activations serve as fixed feature representations fed into a newly trained classifier.

Question 43: What is the hidden state in an RNN?

A vector summarizing information from previous time steps passed to the next step (Correct answer)
The final layer's output probabilities
The weight matrix connecting input to hidden layer
The loss value at each step

Correct answer: A vector summarizing information from previous time steps passed to the next step

The hidden state is a learned vector representation that encodes relevant information from all previous time steps and is updated at each new input.

Question 44: What is the Fréchet Inception Distance (FID) used to measure?

The distance between two points in latent space
Discriminator accuracy in GAN training
Training speed of generative models in FLOPS
The quality and diversity of generated images by comparing statistics of generated and real image feature distributions (Correct answer)

Correct answer: The quality and diversity of generated images by comparing statistics of generated and real image feature distributions

FID measures the Fréchet distance between Inception feature distributions of real and generated images, with lower values indicating more realistic and diverse generated samples.

Question 45: What are the key components of an LSTM cell that vanilla RNNs lack?

Batch normalization and dropout layers
Forget gate, input gate, and output gate controlling information flow (Correct answer)
Attention heads and positional encodings
Convolutional filters and pooling

Correct answer: Forget gate, input gate, and output gate controlling information flow

LSTMs introduce a cell state and three gating mechanisms — forget, input, and output — that regulate what information is retained, added, or read from memory.

Question 46: What problem do residual (skip) connections in ResNet primarily solve?

Slow data loading during training
Excessive memory consumption during inference
Class imbalance in the training data
Degradation of accuracy in very deep networks due to difficult gradient flow (Correct answer)

Correct answer: Degradation of accuracy in very deep networks due to difficult gradient flow

Skip connections let gradients flow directly through identity paths, enabling very deep networks to train without degradation.

Question 47: What is knowledge distillation in deep learning?

Extracting knowledge from datasets using unsupervised methods
Using ensemble predictions to label unlabeled data
Training a smaller student model to mimic the soft output probabilities of a larger teacher model (Correct answer)
Compressing model weights through quantization

Correct answer: Training a smaller student model to mimic the soft output probabilities of a larger teacher model

Knowledge distillation trains a compact student model on the soft probability outputs (dark knowledge) of a large teacher model, transferring richer information than hard labels alone.

Question 48: What is gradient clipping used for when training RNNs?

Capping gradient magnitudes to prevent exploding gradients from destabilizing training (Correct answer)
Preventing the loss from going below zero
Removing neurons with zero gradients
Increasing the learning rate dynamically

Correct answer: Capping gradient magnitudes to prevent exploding gradients from destabilizing training

Gradient clipping rescales gradients when their norm exceeds a threshold, preventing the large weight updates that cause divergence due to exploding gradients in deep or long RNNs.

Question 49: What is the receptive field of a neuron in a CNN?

The spatial size of the output feature map
The region of the input image that influences that neuron's activation (Correct answer)
The number of filters in the layer
The set of all neurons in the same layer

Correct answer: The region of the input image that influences that neuron's activation

The receptive field defines which input pixels contribute to a given neuron's output, growing larger in deeper layers due to successive convolutions.

Question 50: What is 'few-shot learning' in deep learning?

A regularization technique that drops most neurons
Training a model with very few epochs
Using a small neural network with few parameters
A learning paradigm where a model generalizes to new tasks from only a handful of labeled examples (Correct answer)

Correct answer: A learning paradigm where a model generalizes to new tasks from only a handful of labeled examples

Few-shot learning focuses on enabling models to learn new concepts from very few (e.g., 1–5) labeled examples per class, leveraging prior knowledge to make this generalization possible.

Question 51: Which approach is the core idea behind Model-Agnostic Meta-Learning (MAML)?

Using an ensemble of pre-trained models without fine-tuning
Learning an initialization of model parameters that can be quickly adapted to new tasks with few gradient steps (Correct answer)
Replacing gradient descent with evolutionary algorithms
Pre-training a model on a fixed large dataset and never updating it

Correct answer: Learning an initialization of model parameters that can be quickly adapted to new tasks with few gradient steps

MAML meta-trains model parameters to lie in a region of the loss landscape from which a small number of gradient updates on a new task leads to strong performance, enabling fast adaptation.

Question 52: What is 'knowledge distillation' in deep learning?

Removing redundant neurons via pruning
Compressing weights using quantization techniques
Extracting symbolic rules from a neural network for interpretability
Training a smaller student model to mimic the output distribution of a larger teacher model (Correct answer)

Correct answer: Training a smaller student model to mimic the output distribution of a larger teacher model

Knowledge distillation trains a compact student model to match the soft probability outputs (dark knowledge) of a larger, more capable teacher model, transferring learned knowledge without the teacher's computational cost.

Question 53: What is the primary benefit of mixed-precision (FP16/FP32) training on modern GPUs?

Guaranteed elimination of overfitting
Automatic hyperparameter selection
Removal of the need for a loss function
Faster computation and lower memory use with minimal accuracy loss (Correct answer)

Correct answer: Faster computation and lower memory use with minimal accuracy loss

Half-precision arithmetic exploits tensor cores and halves memory traffic while loss scaling preserves numerical stability.

Question 54: What is a GRU (Gated Recurrent Unit) and how does it differ from an LSTM?

A GRU simplifies LSTM by merging cell and hidden state and using only two gates (Correct answer)
A GRU uses attention instead of recurrence
A GRU replaces gradient-based learning with evolutionary methods
A GRU has more gates than an LSTM

Correct answer: A GRU simplifies LSTM by merging cell and hidden state and using only two gates

GRUs combine the forget and input gates into a single update gate and merge the cell and hidden states, reducing parameters while achieving comparable performance to LSTMs on many tasks.

Question 55: What is the vanishing gradient problem particularly severe for in RNNs?

Short sequences of 3–5 tokens
Batch normalization in recurrent layers
Long-range dependencies spanning many time steps (Correct answer)
Single-step predictions

Correct answer: Long-range dependencies spanning many time steps

In RNNs, gradients must be backpropagated through many time steps, causing them to shrink exponentially and preventing the network from learning long-range dependencies.

Question 56: What is an autoencoder and what are its two main components?

A generative model; generator and discriminator
A supervised classifier; feature extractor and classification head
A neural network that compresses input to a low-dimensional latent code and reconstructs it; encoder and decoder (Correct answer)
A sequence model; embedding layer and recurrent layer

Correct answer: A neural network that compresses input to a low-dimensional latent code and reconstructs it; encoder and decoder

An autoencoder maps input to a compressed latent representation via the encoder and reconstructs the input from that representation via the decoder, learning compact data representations.

Question 57: What is teacher forcing in RNN training?

Feeding the ground-truth previous output as the next input during training rather than the model's own prediction (Correct answer)
Using a separate teacher network to guide training
Clipping gradients during backpropagation
Initializing weights from a pretrained language model

Correct answer: Feeding the ground-truth previous output as the next input during training rather than the model's own prediction

Teacher forcing uses the ground-truth token at each step as the next input during training, stabilizing learning by avoiding compounding errors from the model's own predictions.

Question 58: What is latent space interpolation in generative models?

Mixing two training images at the pixel level
Extrapolating beyond the training distribution to novel examples
Smoothly traversing the latent space between two encoded points to generate intermediate samples (Correct answer)
Randomly sampling the latent space to generate diverse outputs

Correct answer: Smoothly traversing the latent space between two encoded points to generate intermediate samples

Latent space interpolation linearly or spherically blends two latent vectors and decodes the intermediate points, producing samples that smoothly transition between the two originals.

Question 59: What distinguishes a generative model from a discriminative model?

Generative models learn the joint distribution p(x,y) and can generate samples; discriminative models learn p(y|x) for classification only (Correct answer)
Generative models produce text; discriminative models produce images
Generative models are always deeper; discriminative models are always shallower
Generative models always use unsupervised learning; discriminative models always use supervised learning

Correct answer: Generative models learn the joint distribution p(x,y) and can generate samples; discriminative models learn p(y|x) for classification only

Generative models model the full data distribution and can synthesize new samples, while discriminative models learn the decision boundary between classes for prediction tasks only.

Question 60: How does an encoder-decoder architecture with attention differ from one without attention?

Attention removes the encoder entirely
Attention allows the decoder to look at all encoder hidden states rather than only a single context vector (Correct answer)
Attention adds convolutional layers to the decoder
Attention replaces the recurrent decoder with a feedforward network

Correct answer: Attention allows the decoder to look at all encoder hidden states rather than only a single context vector

Attention mechanisms let the decoder compute a weighted combination of all encoder hidden states at each decoding step, overcoming the bottleneck of a single fixed context vector.

Question 61: What is the difference between GPT and BERT in terms of architecture and pretraining?

They use identical architectures with different training datasets
GPT uses bidirectional attention; BERT uses unidirectional attention
GPT uses a decoder-only Transformer pretrained with causal language modeling; BERT uses encoder-only with masked language modeling (Correct answer)
GPT is an encoder; BERT is a decoder

Correct answer: GPT uses a decoder-only Transformer pretrained with causal language modeling; BERT uses encoder-only with masked language modeling

GPT is a decoder-only model trained to predict the next token (causal LM), while BERT is an encoder-only model trained with masked tokens, making them suited for generation vs. understanding tasks respectively.

Deep Learning Fundamentals Assessment

A comprehensive assessment of deep learning concepts including neural network architectures, convolutional and recurrent networks, generative models, and practical data science applications. Based on the deeplearning.ai Deep Learning Specialization curriculum structure.

Questions

90 min

Time Limit

70%

To Pass

Checking for saved progress...

Exam Rules

You can skip questions and return to them later
Flag questions for review before submitting
No feedback shown until you submit the entire exam
Unanswered questions count as wrong — answer everything
10 pretest questions are mixed in and don't affect your score
Timer auto-submits when time runs out
Your progress is auto-saved every 30 seconds