AI AI Engineer: Neural Networks and Deep Learning

Question 1

What is the vanishing gradient problem in deep neural networks?

Accepted Answer

Gradients become extremely small, making early layers learn very slowly

Answer

Vanishing gradients occur when backpropagated gradients shrink exponentially through layers, causing weights in early layers to update negligibly.

Question 2

Which activation function is most commonly used in hidden layers of modern deep neural networks to mitigate the vanishing gradient problem?

Accepted Answer

ReLU (Rectified Linear Unit)

Answer

ReLU outputs max(0, x), maintaining gradient magnitude for positive inputs and largely avoiding vanishing gradients compared to sigmoid or tanh.

Question 3

What is the purpose of batch normalization in a neural network?

Accepted Answer

To normalize activations within a layer to stabilize and accelerate training

Answer

Batch normalization normalizes layer inputs to zero mean and unit variance, reducing internal covariate shift and enabling higher learning rates.

Question 4

In a convolutional neural network (CNN), what does 'stride' control?

Accepted Answer

How many pixels the filter moves at each step

Answer

Stride determines how many pixels the convolutional filter shifts at each step, controlling the spatial dimensions of the output feature map.

Question 5

What is dropout regularization in neural networks?

Accepted Answer

Randomly setting a fraction of neurons to zero during training to prevent overfitting

Answer

Dropout randomly deactivates neurons during each training step, forcing the network to learn redundant representations and reducing overfitting.

AI - Engineer Practice Test