Machine Learning Deep Learning 5 — Questions and Answers
Question 1: What is the primary advantage of using a pre-trained word embedding like Word2Vec or GloVe?
- They eliminate the need for any model training
- They provide dense semantic representations that capture word relationships from large corpora (Correct answer)
- They convert text to one-hot vectors efficiently
- They automatically handle out-of-vocabulary words
Correct answer: They provide dense semantic representations that capture word relationships from large corpora
Pretrained embeddings encode semantic similarity (e.g., king − man + woman ≈ queen) learned from billions of words, giving models a strong starting point.
Question 2: In a U-Net architecture, what is the purpose of the skip connections between encoder and decoder?
- To speed up convolution operations
- To pass high-resolution feature maps from the encoder to corresponding decoder layers (Correct answer)
- To apply dropout between encoder and decoder
- To share weights across symmetric layers
Correct answer: To pass high-resolution feature maps from the encoder to corresponding decoder layers
U-Net skip connections concatenate encoder feature maps to decoder inputs, preserving spatial detail lost during downsampling for precise pixel-level predictions.
Question 3: What is model quantization in the context of deploying deep learning models?
- Measuring model accuracy on a held-out test set
- Reducing the numerical precision of weights (e.g., float32 → int8) to speed up inference (Correct answer)
- Splitting a large model across multiple GPUs
- Removing redundant neurons from hidden layers
Correct answer: Reducing the numerical precision of weights (e.g., float32 → int8) to speed up inference
Quantization lowers the bit-width of weights and activations, reducing memory footprint and latency at the cost of a small accuracy drop.
Question 4: What distinguishes a variational autoencoder (VAE) from a standard autoencoder?
- VAE uses a convolutional encoder; standard AE uses fully connected layers
- VAE learns a probabilistic latent space and can generate new samples by sampling from it (Correct answer)
- VAE uses binary cross-entropy; standard AE uses MSE only
- VAE requires labeled data; standard AE is unsupervised
Correct answer: VAE learns a probabilistic latent space and can generate new samples by sampling from it
VAE encodes inputs as distributions (mean and variance) over a latent space, enabling smooth interpolation and generation of new data by sampling.
Question 5: What is knowledge distillation in deep learning?
- Extracting features from unlabeled data using clustering
- Training a smaller student model to mimic the soft outputs of a larger teacher model (Correct answer)
- Compressing weights using principal component analysis
- Transferring weights directly from one model to another
Correct answer: Training a smaller student model to mimic the soft outputs of a larger teacher model
Knowledge distillation trains a compact student network to match the probability outputs of a larger teacher, capturing the teacher's learned knowledge efficiently.
Question 6: What is the receptive field of a neuron in a convolutional neural network?
- The number of filters applied at each layer
- The region of the input image that influences a particular neuron's activation (Correct answer)
- The stride used during the convolution operation
- The number of neurons connected to the output layer
Correct answer: The region of the input image that influences a particular neuron's activation
A neuron's receptive field is the input region it 'sees'; deeper layers have larger receptive fields due to successive convolutions.
Question 7: Which of the following best describes the attention mechanism in neural machine translation?
- It selects a fixed context vector from the encoder for each decoder step
- It allows the decoder to dynamically focus on different encoder states for each output token (Correct answer)
- It replaces the encoder with a simpler bag-of-words representation
- It applies convolutions over the source sentence for each output step
Correct answer: It allows the decoder to dynamically focus on different encoder states for each output token
Attention computes a weighted sum of encoder hidden states for each decoder step, letting the model align output tokens with relevant input positions.
Machine Learning Deep Learning 5
What is the primary advantage of using a pre-trained word embedding like Word2Vec or GloVe?