Deep Learning Neural Network Fundamentals

Question 1

What is the primary purpose of an activation function in a neural network?

Accepted Answer

To introduce non-linearity

Answer

Activation functions introduce non-linearity, enabling neural networks to learn complex patterns beyond simple linear mappings.

Question 2

Which activation function suffers from the 'dying ReLU' problem?

Accepted Answer

ReLU

Answer

ReLU neurons can permanently output zero for all inputs when weights push the input to always be negative, rendering them inactive.

Question 3

What does the term 'vanishing gradient' describe in deep networks?

Accepted Answer

Gradients shrinking near zero during backpropagation

Answer

Vanishing gradients occur when backpropagated error signals shrink exponentially through layers, preventing early layers from learning effectively.

Question 4

Which weight initialization method is specifically designed to work well with ReLU activations?

Accepted Answer

He initialization

Answer

He initialization sets initial weights with variance scaled by 2/n, accounting for the fact that ReLU zeroes out half its inputs.

Question 5

In a feedforward neural network, what direction does information flow during inference?

Accepted Answer

Forward from input to output

Answer

During inference, data passes forward from the input layer through hidden layers to the output layer without any backward pass.

Deep Learning Practice Test