NCA GPU Architecture & Fundamentals 3 — Questions and Answers
Question 1: What is the purpose of Tensor Cores in NVIDIA GPUs like Volta, Turing, Ampere, and Hopper?
- Accelerate ray-triangle intersection tests for real-time rendering
- Perform high-throughput matrix multiply-accumulate (MMA) operations for AI workloads (Correct answer)
- Decode compressed video streams in hardware
- Execute SIMD integer operations for scientific computing
Correct answer: Perform high-throughput matrix multiply-accumulate (MMA) operations for AI workloads
Tensor Cores are specialized hardware units designed to execute mixed-precision matrix multiply-accumulate operations at very high throughput, accelerating deep learning training and inference.
Question 2: In NVIDIA's CUDA programming model, what is the relationship between a thread block and a Streaming Multiprocessor (SM)?
- One SM can process exactly one thread block at a time
- Multiple thread blocks can reside on a single SM simultaneously, subject to resource limits (Correct answer)
- Thread blocks are always distributed across all SMs equally
- Each SM executes exactly one warp from each resident thread block per clock
Correct answer: Multiple thread blocks can reside on a single SM simultaneously, subject to resource limits
Multiple thread blocks can be co-resident on an SM simultaneously, limited by available shared memory, registers, and block slots, enabling latency hiding through concurrency.
Question 3: What is 'memory coalescing' in the context of NVIDIA GPU global memory access?
- Combining multiple L2 cache lines into a single HBM access
- Merging consecutive thread memory accesses into a single wide memory transaction (Correct answer)
- Prefetching data from system RAM into GPU global memory
- Compressing sparse tensors before storing them to VRAM
Correct answer: Merging consecutive thread memory accesses into a single wide memory transaction
Memory coalescing merges individual thread memory requests into a single wide memory transaction when threads in a warp access consecutive addresses, maximizing memory bandwidth utilization.
Question 4: Which NVIDIA interconnect technology allows multiple GPUs to share a unified memory address space with high bandwidth?
- PCIe Gen 5
- NVLink (Correct answer)
- InfiniBand EDR
- RoCE v2
Correct answer: NVLink
NVLink is NVIDIA's high-bandwidth GPU-to-GPU interconnect that supports peer-to-peer memory access and high-throughput data transfers far exceeding PCIe bandwidth.
Question 5: What does 'occupancy' mean in the context of NVIDIA GPU performance tuning?
- The percentage of VRAM currently in use by active kernels
- The ratio of active warps to the maximum possible warps on an SM (Correct answer)
- The fraction of Tensor Cores that are actively performing MMA operations
- The number of concurrent CUDA streams executing on the GPU
Correct answer: The ratio of active warps to the maximum possible warps on an SM
Occupancy is the ratio of active warps to the maximum warps an SM can support, and higher occupancy generally enables better latency hiding through warp switching.
Question 6: In NVIDIA GPU architecture, what is the purpose of the constant memory cache?
- Cache frequently written output tensors to reduce DRAM write traffic
- Provide broadcast-efficient read-only access for data shared by all threads (Correct answer)
- Store the GPU's microcode and firmware permanently
- Cache texture samples from texture units before filtering
Correct answer: Provide broadcast-efficient read-only access for data shared by all threads
Constant memory is optimized for broadcast reads—when all threads in a warp read the same address, it is served in a single cached transaction, making it ideal for kernel parameters and lookup tables.
Question 7: What is the function of NVSwitch in multi-GPU systems like DGX A100?
- Provide dedicated high-speed storage I/O for NVMe SSDs
- Enable all-to-all GPU NVLink connectivity at full bandwidth without bottlenecks (Correct answer)
- Manage power distribution across multiple GPUs in a server rack
- Offload network packet processing from the CPU to the GPU cluster
Correct answer: Enable all-to-all GPU NVLink connectivity at full bandwidth without bottlenecks
NVSwitch is a switching fabric chip that provides full all-to-all NVLink connectivity between all GPUs in a system, enabling any GPU to communicate with any other at full NVLink bandwidth.
NCA GPU Architecture & Fundamentals 3
What is the purpose of Tensor Cores in NVIDIA GPUs like Volta, Turing, Ampere, and Hopper?