NCA Computer Vision & NLP Workloads 3 — Questions and Answers
Question 1: In a multi-GPU NLP training setup, what does the 'tensor parallelism' strategy split across GPUs?
- The dataset batches across GPUs
- Individual weight matrices within a single layer (Correct answer)
- Gradient accumulation steps
- The tokenizer vocabulary
Correct answer: Individual weight matrices within a single layer
Tensor parallelism partitions individual weight matrices (e.g., attention projections) across multiple GPUs, enabling parallelism within a single transformer layer.
Question 2: What is the role of the 'calibration dataset' when applying INT8 quantization in TensorRT?
- It is used to fine-tune the model weights
- It provides representative inputs to compute optimal activation scaling factors (Correct answer)
- It sets the learning rate for quantization-aware training
- It defines the dynamic range of model outputs
Correct answer: It provides representative inputs to compute optimal activation scaling factors
The calibration dataset allows TensorRT to observe real activation distributions and compute per-tensor scaling factors that minimize quantization error.
Question 3: Which attention mechanism variant reduces the memory complexity of self-attention from O(n²) to near-linear, benefiting long-sequence NLP tasks?
- Multi-Head Attention
- Flash Attention (Correct answer)
- Grouped Query Attention
- Cross Attention
Correct answer: Flash Attention
Flash Attention uses tiling and recomputation to avoid materializing the full N×N attention matrix, reducing memory usage to O(n) while maintaining mathematical equivalence.
Question 4: For image segmentation tasks, which NVIDIA pretrained model foundation available in NGC is optimized for rapid fine-tuning on medical and scientific imaging?
- NeMo
- TAO Toolkit SegFormer
- MONAI (Correct answer)
- RAPIDS cuDF
Correct answer: MONAI
MONAI (Medical Open Network for AI) provides domain-specific transforms, pretrained models, and workflows optimized for medical image segmentation.
Question 5: When using NVIDIA's TAO Toolkit for transfer learning on a custom object detection dataset, which step directly precedes model export?
- Dataset annotation with CVAT
- Model pruning (Correct answer)
- Engine calibration
- Vocabulary tokenization
Correct answer: Model pruning
In the TAO workflow, model pruning removes redundant weights after fine-tuning before the model is exported and optimized with TensorRT.
Question 6: Which CUDA memory allocation strategy is recommended for NLP batch inference to minimize allocation overhead when handling variable-length token sequences?
- cudaMalloc per request
- Memory pooling with cudaMallocAsync (Correct answer)
- Pinned host memory only
- Unified virtual memory
Correct answer: Memory pooling with cudaMallocAsync
cudaMallocAsync with CUDA's built-in stream-ordered memory allocator enables efficient pooled allocation that avoids synchronization overhead between requests.
Question 7: In the context of NVIDIA NeMo, what is a 'recipe' used for?
- A curated dataset format for tokenization
- A pre-configured training configuration combining model, data, and optimizer settings (Correct answer)
- A TensorRT calibration script
- An NGC container specification
Correct answer: A pre-configured training configuration combining model, data, and optimizer settings
NeMo recipes are Python scripts that bundle model architecture, dataset, optimizer, and trainer settings into a reproducible, shareable training configuration.
NCA Computer Vision & NLP Workloads 3
In a multi-GPU NLP training setup, what does the 'tensor parallelism' strategy split across GPUs?