Deep Learning Cheat Sheet 2026

The 30 highest-yield Deep Learning facts, distilled from real exam questions. Print it, save it as a PDF, or study it here — free, no sign-up.

1. What does 'stateful' mode mean for an RNN in frameworks like Keras? → The hidden state from the end of one batch is used as the initial state of the next batch
2. What is the primary advantage of transfer learning over training a model from scratch? → It reduces the amount of labeled data and compute required to achieve good performance
3. What problem does batch normalization primarily address? → Internal covariate shift causing unstable training
4. What does an object detection CNN like YOLO do differently from a standard image classification CNN? → It predicts both class labels and bounding box coordinates simultaneously
5. What is a bidirectional RNN? → An RNN that processes the sequence in both forward and backward directions simultaneously
6. In a neural network, what is an epoch? → One complete pass through the entire training dataset
7. In sequence-to-sequence (Seq2Seq) models, what is the role of the encoder? → Compressing the input sequence into a fixed-length context vector
8. What is the computational complexity of self-attention with respect to sequence length n? → O(n²)
9. What is the primary purpose of an activation function in a neural network? → To introduce non-linearity
10. What does fine-tuning mean in the context of transfer learning with pretrained CNNs? → Continuing to train some or all pretrained layers on the new task with a low learning rate
11. What is the hidden state in an RNN? → A vector summarizing information from previous time steps passed to the next step
12. Which of the following strategies is similar to dropout in a neural network in terms of operations? → Bagging
13. In NLP transfer learning, what does BERT's pre-training primarily involve? → Masked language modeling and next sentence prediction
14. What is prompt engineering in the context of large language models? → Crafting input text to elicit desired model behavior without changing model parameters
15. What is a GRU (Gated Recurrent Unit) and how does it differ from an LSTM? → A GRU simplifies LSTM by merging cell and hidden state and using only two gates
16. In the context of large language models, what is autoregressive generation? → Producing one token at a time, conditioning each on all previously generated tokens
17. What is the main limitation of vanilla Seq2Seq models for long input sequences? → The fixed-size context vector loses information when compressing long sequences
18. What is backpropagation in the context of neural network training? → Algorithm to compute gradients via the chain rule
19. The classification error of test data always lowers as the number of hidden layers in a Multi Layer Perceptron increases. Is this statement true or false? → A) False
20. What if we utilize an excessively high learning rate? → Network will not converge
21. What is the vanishing gradient problem's equivalent for very deep CNNs without residual connections? → Gradient signal becoming too weak to update early layer weights effectively
22. What is mode collapse in GAN training? → The generator producing only a few distinct outputs rather than the full data distribution
23. What problem does the forget gate in an LSTM address? → Deciding how much of the previous cell state to retain or discard
24. What is 'discriminative fine-tuning' as used in language model transfer? → Applying different learning rates to different layers, with lower rates for earlier layers
25. Which of the following best describes a hyperparameter in deep learning? → A configuration set before training that controls the learning process
26. What is 'zero-shot learning' in deep learning? → A model's ability to recognize or solve tasks it has never explicitly seen during training
27. Which CNN architecture was the first to demonstrate the power of deep learning on ImageNet, winning in 2012? → AlexNet
28. What is gradient clipping used for when training RNNs? → Capping gradient magnitudes to prevent exploding gradients from destabilizing training
29. What is feature map in the context of a CNN? → The output activation volume produced by applying a set of filters to the previous layer
30. What loss function does a standard GAN generator minimize? → The negative log-probability of the discriminator classifying generated samples as real