Deep Learning and Neural Networks Flashcards

Which type of neural network architecture is specifically designed to handle sequential data such as time series or text?

Answer: Recurrent Neural Network (RNN)

RNNs maintain a hidden state that captures information from previous time steps, making them suited for sequential dependencies.

What is the key innovation of Long Short-Term Memory (LSTM) networks compared to standard RNNs?

Answer: They include gating mechanisms to control information flow and address long-range dependencies

LSTMs use input, forget, and output gates to selectively retain or discard information, solving the vanishing gradient problem in long sequences.

In the context of deep learning, what does 'transfer learning' mean?

Answer: Using a pre-trained model's learned representations as a starting point for a new task

Transfer learning reuses weights from a model trained on a large dataset, fine-tuning them on a smaller target dataset to save time and improve accuracy.

What is the purpose of batch normalization in a neural network?

Answer: To normalize layer inputs to reduce internal covariate shift and speed up training

Batch normalization normalizes activations within each mini-batch, stabilizing training and often allowing higher learning rates.

Which loss function is most appropriate for a multi-class classification problem with a softmax output layer?

Answer: Categorical Cross-Entropy

Categorical cross-entropy measures the divergence between the predicted probability distribution and the true one-hot encoded label across multiple classes.

What does a Generative Adversarial Network (GAN) consist of?

Answer: A generator and a discriminator trained in opposition

GANs pit a generator that creates synthetic samples against a discriminator that distinguishes real from fake, training both through adversarial feedback.