Deep Learning Neural Network Fundamentals 2 — Questions and Answers
Question 1: What is backpropagation in the context of neural network training?
- Forward pass through the network
- Algorithm to compute gradients via the chain rule (Correct answer)
- Method to initialize network weights
- Technique to prune unused neurons
Correct answer: Algorithm to compute gradients via the chain rule
Backpropagation applies the chain rule of calculus to efficiently compute gradients of the loss with respect to every weight in the network.
Question 2: Which loss function is most commonly used for multi-class classification problems?
- Mean Squared Error
- Binary Cross-Entropy
- Categorical Cross-Entropy (Correct answer)
- Hinge Loss
Correct answer: Categorical Cross-Entropy
Categorical Cross-Entropy measures the dissimilarity between predicted probability distributions and one-hot encoded true class labels.
Question 3: What is the universal approximation theorem?
- Any network can approximate any function with infinite layers
- A single hidden layer with enough neurons can approximate any continuous function (Correct answer)
- Deep networks always outperform shallow ones
- Neural networks converge to global optima given enough data
Correct answer: A single hidden layer with enough neurons can approximate any continuous function
The universal approximation theorem states that a feedforward network with one hidden layer and sufficient neurons can approximate any continuous function on a compact subset.
Question 4: What does the learning rate hyperparameter control during gradient descent?
- The number of training epochs
- The size of the step taken in the direction of the negative gradient (Correct answer)
- The number of neurons per layer
- The ratio of training to validation data
Correct answer: The size of the step taken in the direction of the negative gradient
The learning rate determines how large a step is taken when updating weights along the gradient direction, balancing convergence speed and stability.
Question 5: What is a perceptron in neural network terminology?
- A multi-layer network
- A single artificial neuron that computes a weighted sum and applies a threshold (Correct answer)
- A type of convolutional filter
- A normalization technique
Correct answer: A single artificial neuron that computes a weighted sum and applies a threshold
A perceptron is the simplest neural network unit, computing a weighted sum of inputs and producing a binary output via a step activation function.
Question 6: Which of the following best describes a hyperparameter in deep learning?
- A weight learned during training
- A configuration set before training that controls the learning process (Correct answer)
- An output of the final layer
- A gradient computed during backpropagation
Correct answer: A configuration set before training that controls the learning process
Hyperparameters such as learning rate, batch size, and number of layers are set prior to training and govern how the model learns rather than being learned themselves.
Deep Learning Neural Network Fundamentals 2
What is backpropagation in the context of neural network training?