NCA Tensor Core & Mixed Precision 5

Question 1

In quantization-aware training (QAT), how do Tensor Cores enable INT8 inference after FP16/FP32 training?

Accepted Answer

Tensor Cores support INT8 GEMM natively, allowing trained FP32 models to be calibrated and deployed in INT8

Answer

Turing and later Tensor Cores support INT8 GEMM operations, so QAT calibrates scale factors allowing FP32-trained models to run natively in INT8 on Tensor Cores.

Question 2

What is 'loss scaling' and why is it critical specifically for FP16 Tensor Core training?

Accepted Answer

Multiplying the loss by a large scalar so small gradients don't underflow to zero in FP16

Answer

FP16's minimum positive value (~6×10⁻⁸) can cause gradient underflow, so the loss is scaled up before backward pass to shift gradients into representable FP16 range.

Question 3

Which metric best indicates that Tensor Cores are being heavily utilized in a training workload?

Accepted Answer

High sm__pipe_tensor_cycles_active percentage in Nsight Compute

Answer

sm__pipe_tensor_cycles_active measures the fraction of cycles where Tensor Core pipelines are active, directly indicating utilization.

Question 4

Why does using a batch size of 32 instead of 33 often improve Tensor Core efficiency?

Accepted Answer

32 is a multiple of 8, aligning the batch dimension with Tensor Core tile requirements

Answer

Tensor Core tiles require dimensions aligned to multiples of 8 (or 16); batch size 32 aligns perfectly while 33 requires padding that wastes cycles.

Question 5

On Hopper H100, what is the purpose of the Transformer Engine that builds on Tensor Cores?

Accepted Answer

Hardware and software that automatically selects FP8/FP16/BF16 per layer to maximize throughput with minimal accuracy loss

Answer

H100's Transformer Engine dynamically selects the optimal precision (FP8, FP16, BF16) per transformer layer while maintaining accuracy via scaling factors.

(NCA) NVIDIA-Certified Associate Practice Test

(NCA) NVIDIA-Certified Associate Practice Test

NCA Tensor Core & Mixed Precision 5