NCA AI & Deep Learning Frameworks 5 — Questions and Answers
Question 1: What is the attention mechanism in transformer models computing with its Query, Key, and Value matrices?
- Convolving input features across spatial dimensions using learned filters
- Computing weighted sums of value vectors, where weights derive from query-key dot product similarities (Correct answer)
- Applying recurrent state transitions to sequential input tokens
- Normalizing feature distributions across the batch dimension
Correct answer: Computing weighted sums of value vectors, where weights derive from query-key dot product similarities
Self-attention computes dot products between queries and keys to produce attention weights, then uses those weights to aggregate value vectors into context-aware representations.
Question 2: Which NVIDIA software stack component provides a container runtime that allows Docker containers to access GPUs?
- CUDA Toolkit
- NVIDIA Container Toolkit (nvidia-docker) (Correct answer)
- cuDNN
- NVIDIA Driver
Correct answer: NVIDIA Container Toolkit (nvidia-docker)
The NVIDIA Container Toolkit (formerly nvidia-docker2) enables Docker containers to access GPU resources through a container runtime hook.
Question 3: What distinguishes pipeline parallelism from tensor parallelism in distributed training?
- Pipeline parallelism uses multiple nodes while tensor parallelism is single-node only
- Pipeline parallelism splits layers sequentially across GPUs, while tensor parallelism partitions individual layer computations (Correct answer)
- Pipeline parallelism requires gradient checkpointing while tensor parallelism does not
- Pipeline parallelism is only applicable to CNNs while tensor parallelism works with transformers
Correct answer: Pipeline parallelism splits layers sequentially across GPUs, while tensor parallelism partitions individual layer computations
Pipeline parallelism assigns different layers to different GPUs (inter-layer), while tensor parallelism splits individual layer weight matrices across GPUs (intra-layer).
Question 4: In PyTorch, what is `torch.compile()` (introduced in PyTorch 2.0) designed to do?
- Convert PyTorch models to ONNX format for cross-framework deployment
- JIT-compile PyTorch models using TorchInductor to generate optimized code for CPUs and GPUs (Correct answer)
- Automatically apply mixed precision to existing model code
- Parallelize model training across multiple GPUs automatically
Correct answer: JIT-compile PyTorch models using TorchInductor to generate optimized code for CPUs and GPUs
`torch.compile()` uses TorchDynamo and TorchInductor to JIT-compile PyTorch models, often achieving 2x or more speedup through kernel fusion and optimized code generation.
Question 5: What is the purpose of batch normalization in deep neural networks?
- Randomly dropping neurons during training to prevent co-adaptation
- Normalizing layer inputs across the batch dimension to stabilize training and accelerate convergence (Correct answer)
- Constraining weight values to a fixed range to prevent overflow
- Averaging predictions from multiple model checkpoints
Correct answer: Normalizing layer inputs across the batch dimension to stabilize training and accelerate convergence
Batch normalization normalizes activations to have zero mean and unit variance within each mini-batch, reducing internal covariate shift and enabling higher learning rates.
Question 6: Which feature of NVIDIA Multi-Instance GPU (MIG) technology benefits AI inference workloads?
- Combining multiple physical GPUs into a single logical GPU for large models
- Partitioning a single GPU into isolated instances with guaranteed compute and memory resources (Correct answer)
- Enabling FP64 precision on consumer-grade GPUs
- Automatically balancing inference loads across multiple data center GPUs
Correct answer: Partitioning a single GPU into isolated instances with guaranteed compute and memory resources
MIG partitions a single A100 or H100 GPU into up to 7 isolated instances, each with dedicated SM partitions and memory, enabling multiple inference workloads to run securely side-by-side.
Question 7: What problem does the Adam optimizer address that vanilla SGD with momentum does not?
- Second-order curvature estimation using the Hessian matrix
- Adaptive per-parameter learning rates based on first and second moment estimates of gradients (Correct answer)
- Automatic learning rate scheduling based on validation loss plateaus
- Gradient clipping to prevent exploding updates in deep networks
Correct answer: Adaptive per-parameter learning rates based on first and second moment estimates of gradients
Adam maintains exponential moving averages of both gradients (first moment) and squared gradients (second moment) to compute adaptive per-parameter learning rates.
NCA AI & Deep Learning Frameworks 5
What is the attention mechanism in transformer models computing with its Query, Key, and Value matrices?