NCA GPU Architecture & Fundamentals 4 — Questions and Answers
Question 1: What is the primary advantage of High Bandwidth Memory (HBM) over GDDR memory in data center GPUs?
- HBM operates at a higher clock frequency, improving latency-sensitive workloads
- HBM uses a much wider memory bus (1024+ bits), delivering significantly higher bandwidth (Correct answer)
- HBM stores data non-volatilely, eliminating the need for checkpointing
- HBM supports larger individual capacity per chip than any GDDR variant
Correct answer: HBM uses a much wider memory bus (1024+ bits), delivering significantly higher bandwidth
HBM uses a very wide memory bus (1024 bits or more per stack) combined with stacked DRAM dies, achieving memory bandwidths far exceeding GDDR despite lower clock speeds.
Question 2: What is a 'thread block cluster' introduced in NVIDIA Hopper architecture?
- A group of SMs that share an enlarged unified L1 cache
- A collection of thread blocks that can cooperate and share data via distributed shared memory (Correct answer)
- An abstraction for grouping multiple CUDA streams into a single scheduling unit
- A hardware feature for automatically fusing consecutive kernel launches
Correct answer: A collection of thread blocks that can cooperate and share data via distributed shared memory
Thread block clusters in Hopper allow thread blocks on different SMs within a GPC to cooperate using Distributed Shared Memory, enabling new levels of inter-SM data sharing.
Question 3: What does the NVIDIA GPU's register file store, and why is it significant for performance?
- It stores kernel binary code and constants for fast instruction fetch
- It stores per-thread local variables with zero-latency access, enabling fast arithmetic (Correct answer)
- It caches the most recently accessed global memory pages to reduce DRAM traffic
- It holds the warp scheduler state and PC registers for active warps
Correct answer: It stores per-thread local variables with zero-latency access, enabling fast arithmetic
The register file stores per-thread variables and provides single-cycle access latency, making register usage critical for achieving peak arithmetic throughput without memory stalls.
Question 4: Why does 'register spilling' hurt GPU kernel performance?
- It forces the warp scheduler to evict thread blocks from the SM prematurely
- It causes excess register values to be stored in local (global) memory, adding high-latency accesses (Correct answer)
- It disables Tensor Core operations in affected SMs for the duration of the kernel
- It increases power consumption, triggering GPU throttling mechanisms
Correct answer: It causes excess register values to be stored in local (global) memory, adding high-latency accesses
When a kernel uses more registers than available per thread, excess values spill to local memory (backed by global DRAM), incurring hundreds of cycles of latency per access.
Question 5: What is the role of the CUDA Cooperative Groups programming model with respect to GPU synchronization?
- It replaces warp-level SIMT execution with a more flexible MIMD execution model
- It provides flexible, programmer-defined synchronization scopes beyond a single thread block (Correct answer)
- It automatically partitions thread blocks into warps optimally based on data access patterns
- It enables CPU threads to synchronize directly with GPU warps during kernel execution
Correct answer: It provides flexible, programmer-defined synchronization scopes beyond a single thread block
Cooperative Groups extends CUDA synchronization beyond __syncthreads(), allowing developer-defined groups at warp, block, multi-block, or grid scope for more expressive parallel algorithms.
Question 6: In NVIDIA GPUs, what is a 'bank conflict' in shared memory and how does it affect performance?
- Multiple warps reading the same cache line in L2, causing serialized DRAM fetches
- Multiple threads in a warp simultaneously accessing different addresses in the same shared memory bank, causing serialized access (Correct answer)
- Two thread blocks competing for the same shared memory allocation on an SM
- A condition where a warp stalls because its shared memory request conflicts with an L1 eviction
Correct answer: Multiple threads in a warp simultaneously accessing different addresses in the same shared memory bank, causing serialized access
Shared memory is divided into banks, and when multiple threads in a warp access different addresses that map to the same bank, the accesses are serialized, reducing effective shared memory bandwidth.
Question 7: What is Multi-Instance GPU (MIG) technology introduced with NVIDIA A100, and what problem does it solve?
- It allows a single GPU to drive multiple independent display outputs simultaneously
- It partitions a single GPU into isolated instances with dedicated memory and compute, enabling safe multi-tenant workloads (Correct answer)
- It merges multiple physical GPUs into a single logical GPU for large model training
- It enables dynamic switching between compute and graphics workloads on the same GPU
Correct answer: It partitions a single GPU into isolated instances with dedicated memory and compute, enabling safe multi-tenant workloads
MIG partitions a single A100/H100 GPU into up to 7 isolated GPU instances, each with dedicated SM slices and HBM partitions, providing performance isolation for multi-tenant cloud deployments.
NCA GPU Architecture & Fundamentals 4
What is the primary advantage of High Bandwidth Memory (HBM) over GDDR memory in data center GPUs?