NCA Computer Vision & NLP Workloads 3

Question 1

In a multi-GPU NLP training setup, what does the 'tensor parallelism' strategy split across GPUs?

Accepted Answer

Individual weight matrices within a single layer

Answer

Tensor parallelism partitions individual weight matrices (e.g., attention projections) across multiple GPUs, enabling parallelism within a single transformer layer.

Question 2

What is the role of the 'calibration dataset' when applying INT8 quantization in TensorRT?

Accepted Answer

It provides representative inputs to compute optimal activation scaling factors

Answer

The calibration dataset allows TensorRT to observe real activation distributions and compute per-tensor scaling factors that minimize quantization error.

Question 3

Which attention mechanism variant reduces the memory complexity of self-attention from O(n²) to near-linear, benefiting long-sequence NLP tasks?

Accepted Answer

Flash Attention

Answer

Flash Attention uses tiling and recomputation to avoid materializing the full N×N attention matrix, reducing memory usage to O(n) while maintaining mathematical equivalence.

Question 4

For image segmentation tasks, which NVIDIA pretrained model foundation available in NGC is optimized for rapid fine-tuning on medical and scientific imaging?

Accepted Answer

MONAI

Answer

MONAI (Medical Open Network for AI) provides domain-specific transforms, pretrained models, and workflows optimized for medical image segmentation.

Question 5

When using NVIDIA's TAO Toolkit for transfer learning on a custom object detection dataset, which step directly precedes model export?

Accepted Answer

Model pruning

Answer

In the TAO workflow, model pruning removes redundant weights after fine-tuning before the model is exported and optimized with TensorRT.

(NCA) NVIDIA-Certified Associate Practice Test

(NCA) NVIDIA-Certified Associate Practice Test

NCA Computer Vision & NLP Workloads 3