NCA Tensor Core & Mixed Precision 3 — Questions and Answers
Question 1: Ampere Tensor Cores added support for which sparse computation technique that can double effective throughput?
- Block sparsity
- 2:4 structured sparsity (Correct answer)
- Unstructured random sparsity
- Column sparsity
Correct answer: 2:4 structured sparsity
Ampere introduced 2:4 structured sparsity, where exactly 2 of every 4 weights are zero, doubling throughput via the Sparsity feature.
Question 2: When using GradScaler with AMP in PyTorch, what problem does it solve?
- Gradient explosion in FP32
- Gradient underflow (vanishing gradients) in FP16 (Correct answer)
- Slow GPU memory allocation
- Incompatibility between cuBLAS and cuDNN
Correct answer: Gradient underflow (vanishing gradients) in FP16
GradScaler scales the loss before backward pass to prevent FP16 gradients from underflowing to zero during backpropagation.
Question 3: Which precision format introduced in Hopper GPUs is specifically optimized for transformer inference?
- FP8 (Correct answer)
- INT4
- TF32
- FP16
Correct answer: FP8
Hopper (H100) added FP8 precision (E4M3 and E5M2 variants) Tensor Core support, targeting high-throughput transformer inference.
Question 4: What does TF32 (TensorFloat-32) format use compared to standard FP32?
- Same exponent, reduced mantissa (10 bits) (Correct answer)
- Reduced exponent, same mantissa
- Same bit width with different encoding
- Double exponent, half mantissa
Correct answer: Same exponent, reduced mantissa (10 bits)
TF32 keeps FP32's 8-bit exponent but reduces the mantissa to 10 bits, matching FP16 precision while retaining FP32 dynamic range.
Question 5: In multi-GPU training, which NVIDIA library uses Tensor Cores for the all-reduce collective communication pattern?
- NCCL (Correct answer)
- MPI
- NVLINK
- cuBLAS
Correct answer: NCCL
NCCL (NVIDIA Collective Communications Library) implements optimized all-reduce operations that leverage NVLink bandwidth, though compute remains on Tensor Cores via training kernels.
Question 6: Which NVIDIA profiling tool shows Tensor Core utilization as a specific metric during training?
- nvidia-smi
- Nsight Systems
- Nsight Compute (Correct answer)
- cuobjdump
Correct answer: Nsight Compute
Nsight Compute provides kernel-level metrics including sm__pipe_tensor_cycles_active which directly measures Tensor Core utilization.
Question 7: What is the key difference between cuDNN's IMPLICIT_PRECOMP_GEMM and GEMM convolution algorithms regarding Tensor Core use?
- Only GEMM converts convolutions to matrix multiplications for Tensor Core dispatch (Correct answer)
- Only IMPLICIT_PRECOMP_GEMM can use Tensor Cores
- Both are identical in Tensor Core dispatch
- Neither uses Tensor Cores
Correct answer: Only GEMM converts convolutions to matrix multiplications for Tensor Core dispatch
GEMM explicitly converts the convolution to a large matrix multiplication (im2col) which can be dispatched to Tensor Cores via cuBLAS.
NCA Tensor Core & Mixed Precision 3
Ampere Tensor Cores added support for which sparse computation technique that can double effective throughput?