AI Engineer: Neural Networks and Deep Learning Flashcards

What distinguishes a generative adversarial network (GAN) from a variational autoencoder (VAE)?

Answer: GANs use an adversarial training loop between generator and discriminator; VAEs optimize a variational lower bound

GANs pit a generator against a discriminator in an adversarial game, while VAEs learn a probabilistic latent space by maximizing an evidence lower bound (ELBO).

What is 'hyperparameter tuning' in deep learning?

Answer: Searching for optimal settings like learning rate, batch size, and architecture before/during training

Hyperparameter tuning involves systematically searching over configuration choices (learning rate, depth, dropout rate) that are set before training, not learned.

In convolutional neural networks, what is the primary purpose of a pooling layer?

Answer: To reduce spatial dimensions and provide translation invariance

Pooling (e.g., max pooling) downsamples feature maps, reducing computation and providing some invariance to small spatial translations.

What is 'early stopping' as a regularization technique in deep learning?

Answer: Halting training when validation loss stops improving to prevent overfitting

Early stopping monitors validation loss and stops training when it starts increasing, saving the model at the point of best generalization.

Which technique is used to visualize which parts of an input image most influence a CNN's classification decision?

Answer: Grad-CAM (Gradient-weighted Class Activation Mapping)

Grad-CAM uses gradients flowing into the final convolutional layer to produce a heatmap highlighting the regions most important to the prediction.

What is 'knowledge distillation' in the context of deep learning model compression?

Answer: Training a small 'student' model to mimic the outputs of a large 'teacher' model

Knowledge distillation transfers knowledge from a large teacher model to a compact student model by training the student on soft probability outputs of the teacher.