Deep Learning Neural Network Fundamentals

Question 1

What is the primary purpose of an activation function in a neural network?

Accepted Answer

To introduce non-linearity

Answer

To initialize weights

Answer

To normalize inputs

Answer

To reduce overfitting

Question 2

Which activation function suffers from the 'dying ReLU' problem?

Accepted Answer

ReLU

Answer

Sigmoid

Answer

Tanh

Answer

Softmax

Question 3

What does the term 'vanishing gradient' describe in deep networks?

Accepted Answer

Gradients shrinking near zero during backpropagation

Answer

Loss becoming too large

Answer

Weights growing unboundedly

Answer

Activations saturating at high values

Question 4

Which weight initialization method is specifically designed to work well with ReLU activations?

Accepted Answer

He initialization

Answer

Xavier initialization

Answer

Zero initialization

Answer

Random uniform initialization

Question 5

In a feedforward neural network, what direction does information flow during inference?

Accepted Answer

Forward from input to output

Answer

Backward through layers

Answer

Bidirectionally through all layers

Answer

Only through skip connections

Question 6

What is the role of bias terms in a neural network neuron?

Accepted Answer

Shift the activation function horizontally

Answer

Scale the activation output

Answer

Normalize the weight values

Answer

Connect to skip connections