NCA Tensor Core & Mixed Precision 2 — Questions and Answers
Question 1: Which NVIDIA GPU architecture first introduced Tensor Cores?
- Pascal
- Volta (Correct answer)
- Maxwell
- Kepler
Correct answer: Volta
Tensor Cores were introduced in the Volta architecture (V100) in 2017.
Question 2: In the context of Tensor Cores, what does the term 'mixed precision' mean?
- Using different GPUs for different layers
- Performing multiply-accumulate in lower precision with accumulation in higher precision (Correct answer)
- Mixing CPU and GPU computations
- Alternating between INT8 and FP64 every epoch
Correct answer: Performing multiply-accumulate in lower precision with accumulation in higher precision
Mixed precision uses FP16 (or BF16) for multiply operations and FP32 for accumulation to balance speed and numerical stability.
Question 3: What native matrix multiplication shape does a single Volta Tensor Core operation compute?
- 8×8×8
- 16×16×16
- 4×4×4 (Correct answer)
- 32×32×32
Correct answer: 4×4×4
Each Volta Tensor Core performs a 4×4×4 matrix multiply-accumulate per clock cycle.
Question 4: Which PyTorch function enables automatic mixed precision (AMP) training with Tensor Cores?
- torch.cuda.amp.autocast() (Correct answer)
- torch.enable_grad()
- torch.backends.cudnn.benchmark
- torch.nn.DataParallel()
Correct answer: torch.cuda.amp.autocast()
torch.cuda.amp.autocast() automatically casts operations to FP16 where safe, enabling Tensor Core acceleration.
Question 5: Why must tensor dimensions often be multiples of 8 (or 16) to fully utilize Tensor Cores?
- GPU memory is allocated in 8-byte pages
- Tensor Cores operate on fixed tile sizes and padding wastes cycles (Correct answer)
- CUDA warp size is 8 threads
- PyTorch batch size defaults to 8
Correct answer: Tensor Cores operate on fixed tile sizes and padding wastes cycles
Tensor Cores process fixed-size matrix tiles, so non-aligned dimensions cause padding that wastes compute cycles.
Question 6: What is the primary advantage of BF16 over FP16 for Tensor Core training?
- Higher throughput per clock cycle
- Larger dynamic range matching FP32 (Correct answer)
- Smaller memory footprint
- Native support on Pascal GPUs
Correct answer: Larger dynamic range matching FP32
BF16 shares FP32's 8-bit exponent, giving the same dynamic range and reducing gradient overflow issues compared to FP16's 5-bit exponent.
Question 7: Which CUDA library provides GPU-optimized GEMM routines that automatically dispatch to Tensor Cores?
- cuSPARSE
- cuBLAS (Correct answer)
- cuDNN
- NCCL
Correct answer: cuBLAS
cuBLAS provides highly optimized General Matrix Multiplication (GEMM) that dispatches to Tensor Cores on supported hardware.
NCA Tensor Core & Mixed Precision 2
Which NVIDIA GPU architecture first introduced Tensor Cores?