Data Science Deep Learning and Neural Networks 3

Question 1

What is the 'attention mechanism' in deep learning primarily used for?

Accepted Answer

Allowing the model to focus on relevant parts of the input when producing each output

Answer

Reducing the number of layers in a network

Answer

Replacing dropout for regularization

Answer

Scheduling the learning rate during training

Question 2

In deep learning, what is a hyperparameter?

Accepted Answer

A configuration value set before training that controls the learning process

Answer

A weight learned from data during backpropagation

Answer

An output neuron in the final layer

Answer

A normalized input feature

Question 3

What problem does the ReLU activation function help solve compared to sigmoid?

Accepted Answer

Vanishing gradients in deep networks by maintaining non-zero gradients for positive inputs

Answer

Underfitting due to too few parameters

Answer

High computational cost of matrix multiplication

Answer

Class imbalance in training data

Question 4

What distinguishes a deep neural network from a shallow one?

Accepted Answer

Deep networks contain multiple hidden layers enabling hierarchical feature learning

Answer

Deep networks use only linear transformations

Answer

Deep networks require less data to train

Answer

Deep networks do not use backpropagation

Question 5

Which technique is used to prevent overfitting by penalizing large weights in a neural network's loss function?

Accepted Answer

L2 regularization (weight decay)

Answer

Data augmentation

Answer

Gradient clipping

Answer

Learning rate warmup

Question 6

In a neural network, what is the purpose of the softmax function in the output layer for classification tasks?

Accepted Answer

To convert raw logits into a probability distribution that sums to 1

Answer

To clip output values between -1 and 1

Answer

To apply non-linearity in the hidden layers

Answer

To initialize weights using a uniform distribution