Artificial Intelligence Neural Networks and Deep Learning 2 — Questions and Answers
Question 1: What is batch normalization designed to do in deep neural networks?
- Randomly drop neurons to prevent overfitting
- Normalize activations within a layer to stabilize and accelerate training (Correct answer)
- Reduce the number of parameters in the network
- Initialize weights to small random values
Correct answer: Normalize activations within a layer to stabilize and accelerate training
Batch normalization normalizes layer inputs over a mini-batch, reducing internal covariate shift and allowing higher learning rates.
Question 2: Which architecture introduced residual (skip) connections to train very deep networks?
- VGG
- AlexNet
- ResNet (Correct answer)
- Inception
Correct answer: ResNet
ResNet introduced skip connections that add the input of a block to its output, enabling gradients to flow more easily through hundreds of layers.
Question 3: What distinguishes a recurrent neural network (RNN) from a feedforward network?
- RNNs use convolutional filters
- RNNs have feedback connections that allow information to persist across time steps (Correct answer)
- RNNs cannot process sequences
- RNNs always use attention mechanisms
Correct answer: RNNs have feedback connections that allow information to persist across time steps
RNNs have recurrent connections that pass hidden states from one time step to the next, enabling them to model sequential and temporal data.
Question 4: What is the primary advantage of using Long Short-Term Memory (LSTM) over a simple RNN?
- LSTMs are faster to train
- LSTMs can capture long-range dependencies by controlling information flow with gates (Correct answer)
- LSTMs require less memory
- LSTMs do not need backpropagation
Correct answer: LSTMs can capture long-range dependencies by controlling information flow with gates
LSTM gates (input, forget, output) regulate what information is stored, forgotten, or passed on, mitigating the vanishing gradient problem in long sequences.
Question 5: In a softmax output layer for multi-class classification, what do the output values represent?
- Raw logit scores for each class
- A probability distribution over all classes summing to 1 (Correct answer)
- Binary indicators of class membership
- Normalized feature values
Correct answer: A probability distribution over all classes summing to 1
Softmax converts raw logits into a probability distribution where all values are positive and sum to 1, one per class.
Question 6: What is transfer learning in deep learning?
- Moving training between different hardware
- Using a pre-trained model's learned weights as a starting point for a new task (Correct answer)
- Training a model on multiple datasets simultaneously
- Transferring data between cloud storage providers
Correct answer: Using a pre-trained model's learned weights as a starting point for a new task
Transfer learning reuses weights from a model trained on a large dataset (e.g., ImageNet) as initialization for a related task, reducing training time and data requirements.
Artificial Intelligence Neural Networks and Deep Learning 2
What is batch normalization designed to do in deep neural networks?