Artificial Intelligence Neural Networks 1 — Questions and Answers
Question 1: What is the role of an activation function in a neural network?
- Initialize the weights
- Introduce non-linearity into the model (Correct answer)
- Calculate the loss
- Normalize the input data
Correct answer: Introduce non-linearity into the model
Activation functions add non-linearity, allowing neural networks to learn complex patterns beyond linear mappings.
Question 2: Which activation function outputs values between 0 and 1 and is commonly used in binary classification output layers?
- ReLU
- Tanh
- Sigmoid (Correct answer)
- Softmax
Correct answer: Sigmoid
The sigmoid function maps any real number to (0,1), making it suitable for binary probability outputs.
Question 3: What is backpropagation in neural network training?
- Feeding data forward through the network
- Using the chain rule to compute gradients and update weights (Correct answer)
- Randomly initializing network weights
- Normalizing layer activations
Correct answer: Using the chain rule to compute gradients and update weights
Backpropagation computes gradients of the loss with respect to all weights by applying the chain rule backwards through the network.
Question 4: What is a 'vanishing gradient' problem in deep neural networks?
- Gradients become extremely large, destabilizing training
- Gradients shrink to near zero as they propagate back, slowing learning in early layers (Correct answer)
- The network loses memory of earlier inputs
- Weights are initialized too close to zero
Correct answer: Gradients shrink to near zero as they propagate back, slowing learning in early layers
Vanishing gradients occur when gradients become too small during backpropagation, preventing early layers from learning effectively.
Question 5: Which layer type in a neural network is fully connected, meaning every input neuron links to every output neuron?
- Convolutional layer
- Pooling layer
- Dense (fully connected) layer (Correct answer)
- Embedding layer
Correct answer: Dense (fully connected) layer
A dense layer connects every neuron in the previous layer to every neuron in the next layer.
Question 6: What does 'dropout' do during neural network training?
- Removes neurons permanently to reduce model size
- Randomly deactivates neurons during each training step to reduce overfitting (Correct answer)
- Scales learning rate based on layer depth
- Normalizes output of each layer
Correct answer: Randomly deactivates neurons during each training step to reduce overfitting
Dropout randomly sets a fraction of neuron activations to zero during training, acting as a regularization technique.
Artificial Intelligence Neural Networks 1
What is the role of an activation function in a neural network?