AI Engineer: Neural Networks and Deep Learning Flashcards

What is a recurrent neural network (RNN) primarily designed to handle?

Answer: Sequential and time-series data with temporal dependencies

RNNs maintain a hidden state across time steps, making them suited for sequential data like text, audio, and time series.

What problem do LSTM (Long Short-Term Memory) networks solve compared to vanilla RNNs?

Answer: Long-range dependency learning by mitigating vanishing gradients over long sequences

LSTMs use gating mechanisms (input, forget, output gates) to selectively retain or discard information, enabling learning of long-range dependencies.

In deep learning, what is transfer learning?

Answer: Reusing a model pretrained on a large dataset as a starting point for a new task

Transfer learning leverages knowledge from a pretrained model (e.g., ImageNet-trained ResNet) by fine-tuning it on a smaller task-specific dataset.

What is the role of the 'softmax' activation function in a multi-class classification output layer?

Answer: Converts raw logits into a probability distribution summing to 1

Softmax exponentiates each logit and divides by the sum of all exponentiated logits, producing class probabilities that sum to 1.

Which deep learning architecture introduced skip (residual) connections to enable training of very deep networks?

Answer: ResNet

ResNet introduced residual connections that add layer inputs directly to outputs, allowing gradients to flow more easily and enabling hundreds of layers.

What is the purpose of weight initialization in neural networks?

Answer: To set initial parameter values that promote stable gradient flow at the start of training

Good weight initialization (e.g., Xavier, He) prevents vanishing or exploding gradients from the first forward pass, enabling stable training.