Neural Network Cheat Sheet 2026

The 30 highest-yield Neural Network facts, distilled from real exam questions. Print it, save it as a PDF, or study it here — free, no sign-up.

1. What is the role of the validation set during neural network training? → To tune hyperparameters and monitor for overfitting without contaminating the test set
2. What does max pooling accomplish in a CNN? → It downsamples feature maps by taking the maximum value in each pooling window
3. Which of the following is a genetic algorithm equivalent? → Neural network
4. What distinguishes a perceptron? → a single layer feed-forward neural network with pre-processing
5. What is the vanishing gradient problem in deep neural networks? → Gradients become extremely small and fail to update early layers
6. What does the strength of association rule that is indicated by support and → Confidence factor
7. Which phenomenon occurs when a neural network's training loss is low but its generalization gap (train loss minus test loss) is large? → Overfitting
8. What does L2 regularization (weight decay) do to the weights during training? → It encourages weights to remain small by penalizing their squared magnitude
9. What is the primary use case for TensorFlow Lite? → Running pretrained neural network models on mobile and embedded devices with low latency
10. Which architecture is most commonly used as the backbone for object detection models like YOLO and Faster R-CNN? → Convolutional neural network
11. What is curriculum learning in neural network training? → Training the network on easier examples first, then gradually increasing difficulty
12. What is the softmax function used for in neural networks? → To convert raw logits into a probability distribution over classes
13. What is a saddle point in the loss landscape of a neural network? → A critical point that is a minimum in some directions and a maximum in others
14. Data augmentation helps reduce overfitting primarily by: → Artificially expanding the training dataset with transformed samples
15. An example of which kind performs predictive modeling is → Data Mining process
16. What problem does batch normalization primarily address in deep networks? → Internal covariate shift between layers
17. What is a bidirectional RNN? → An RNN that processes data both forwards and backwards in time simultaneously
18. What is the key innovation of DenseNet compared to ResNet? → Connecting every layer to every subsequent layer
19. What is the output range of the sigmoid activation function? → (0, 1)
20. What distinguishes an automated vehicle? → Supervised learning
21. Which component of a Variational Autoencoder (VAE) enables backpropagation through the sampling step? → The reparameterization trick
22. In mini-batch gradient descent, what happens to the gradient estimate as batch size increases? → It becomes a more accurate estimate of the true gradient
23. Which of the following is a symptom of underfitting in a neural network? → Low training accuracy and low test accuracy
24. In machine learning, what does not refer to a neural network layer → Bias layer
25. Choose an application for decision trees in machine learning. → Classification
26. What is a convolutional layer and what operation does it perform? → A layer that slides learned filters across the input to produce feature maps
27. In which scenario would you use gradient accumulation during training? → When GPU memory is too small to fit large effective batch sizes
28. What does the learning rate control in neural network training? → The step size taken in the direction of the negative gradient
29. What is a 1×1 convolution and why is it useful? → A convolution that mixes channel information without affecting spatial dimensions
30. What is residual learning in the context of deep neural networks? → Adding skip connections so layers learn residual mappings