Deep Learning Cheat Sheet 2026

The 30 highest-yield Deep Learning facts, distilled from real exam questions. Print it, save it as a PDF, or study it here — free, no sign-up.

1. What does 'stateful' mode mean for an RNN in frameworks like Keras? → The hidden state from the end of one batch is used as the initial state of the next batch
2. What is the primary advantage of transfer learning over training a model from scratch? → It reduces the amount of labeled data and compute required to achieve good performance
3. What problem does batch normalization primarily address? → Internal covariate shift causing unstable training
4. Why is ReLU generally preferred over sigmoid for hidden layers in deep networks? → ReLU avoids saturation for positive inputs, preserving gradient magnitude in deep stacks
5. What does an object detection CNN like YOLO do differently from a standard image classification CNN? → It predicts both class labels and bounding box coordinates simultaneously
6. What is a bidirectional RNN? → An RNN that processes the sequence in both forward and backward directions simultaneously
7. In a neural network, what is an epoch? → One complete pass through the entire training dataset
8. In sequence-to-sequence (Seq2Seq) models, what is the role of the encoder? → Compressing the input sequence into a fixed-length context vector
9. A data scientist deploys a fraud-detection network trained on 2022 data and observes accuracy decay in production during 2024. What is this phenomenon called? → Data drift (distribution shift)
10. What is the computational complexity of self-attention with respect to sequence length n? → O(n²)
11. What is the primary purpose of an activation function in a neural network? → To introduce non-linearity
12. What does fine-tuning mean in the context of transfer learning with pretrained CNNs? → Continuing to train some or all pretrained layers on the new task with a low learning rate
13. In a GAN, the generator produces nearly identical outputs regardless of the input noise vector. What is this failure mode called? → Mode collapse
14. What is the hidden state in an RNN? → A vector summarizing information from previous time steps passed to the next step
15. Which of the following strategies is similar to dropout in a neural network in terms of operations? → Bagging
16. In NLP transfer learning, what does BERT's pre-training primarily involve? → Masked language modeling and next sentence prediction
17. What is prompt engineering in the context of large language models? → Crafting input text to elicit desired model behavior without changing model parameters
18. What is a GRU (Gated Recurrent Unit) and how does it differ from an LSTM? → A GRU simplifies LSTM by merging cell and hidden state and using only two gates
19. In the context of large language models, what is autoregressive generation? → Producing one token at a time, conditioning each on all previously generated tokens
20. What is the main limitation of vanilla Seq2Seq models for long input sequences? → The fixed-size context vector loses information when compressing long sequences
21. Which method explains an individual prediction of a deep model by attributing importance scores to input features? → SHAP or integrated gradients
22. What is backpropagation in the context of neural network training? → Algorithm to compute gradients via the chain rule
23. The classification error of test data always lowers as the number of hidden layers in a Multi Layer Perceptron increases. Is this statement true or false? → A) False
24. What if we utilize an excessively high learning rate? → Network will not converge
25. Which practice most directly reduces the memory footprint and latency of a trained network for edge deployment? → Quantization of weights to lower-precision formats such as INT8
26. What is the vanishing gradient problem's equivalent for very deep CNNs without residual connections? → Gradient signal becoming too weak to update early layer weights effectively
27. Which optimizer combines momentum with per-parameter adaptive learning rates derived from first and second moment estimates? → Adam
28. What is mode collapse in GAN training? → The generator producing only a few distinct outputs rather than the full data distribution
29. What problem do residual (skip) connections in ResNet primarily solve? → Degradation of accuracy in very deep networks due to difficult gradient flow
30. What problem does the forget gate in an LSTM address? → Deciding how much of the previous cell state to retain or discard