Artificial Intelligence Neural Networks and Deep Learning 1 — Questions and Answers
Question 1: What is the activation function that outputs values strictly between 0 and 1, commonly used in binary classification output layers?
- ReLU
- Tanh
- Sigmoid (Correct answer)
- Softmax
Correct answer: Sigmoid
The sigmoid function squashes any real-valued input to the (0, 1) range, making it useful for binary probability outputs.
Question 2: Which layer type in a convolutional neural network applies a learned filter to detect local patterns in an image?
- Fully connected layer
- Pooling layer
- Convolutional layer (Correct answer)
- Dropout layer
Correct answer: Convolutional layer
A convolutional layer slides learned filters over the input to produce feature maps capturing local spatial patterns like edges and textures.
Question 3: What problem does the 'vanishing gradient' phenomenon cause in deep neural networks?
- The model converges too quickly
- Gradients become very small in early layers, halting learning (Correct answer)
- The network generates random outputs
- Training data is too small to support the model
Correct answer: Gradients become very small in early layers, halting learning
Vanishing gradients occur when backpropagated error signals shrink exponentially through many layers, making early weights nearly untrainable.
Question 4: What is the purpose of a dropout layer in a neural network?
- To speed up forward passes
- To randomly deactivate neurons during training to reduce overfitting (Correct answer)
- To normalize activations across a batch
- To increase the number of parameters
Correct answer: To randomly deactivate neurons during training to reduce overfitting
Dropout randomly sets a fraction of neuron activations to zero during each training step, acting as an ensemble regularization technique.
Question 5: Which optimizer adapts the learning rate for each parameter based on historical gradient information?
- SGD
- Momentum
- Adam (Correct answer)
- Batch gradient descent
Correct answer: Adam
Adam (Adaptive Moment Estimation) maintains per-parameter adaptive learning rates using estimates of first and second moments of gradients.
Question 6: What does 'backpropagation' compute in a neural network training cycle?
- The forward pass predictions
- The gradient of the loss with respect to each network weight (Correct answer)
- The optimal batch size for training
- The number of layers needed
Correct answer: The gradient of the loss with respect to each network weight
Backpropagation applies the chain rule to efficiently compute the gradient of the loss with respect to every weight, enabling gradient-based updates.
Artificial Intelligence Neural Networks and Deep Learning 1
What is the activation function that outputs values strictly between 0 and 1, commonly used in binary classification output layers?