NCA GPU Architecture & Fundamentals 2 — Questions and Answers
Question 1: What is the primary role of the L1 cache in NVIDIA GPU Streaming Multiprocessors (SMs)?
- Store long-term model weights persistently
- Provide low-latency data access for threads within an SM (Correct answer)
- Synchronize data between the CPU and GPU
- Cache outputs from the display pipeline
Correct answer: Provide low-latency data access for threads within an SM
The L1 cache in each SM provides fast, low-latency access to frequently used data for threads executing within that SM.
Question 2: In NVIDIA GPU architecture, what does the term 'warp' refer to?
- A group of 64 CUDA cores executing in parallel
- A group of 32 threads that execute the same instruction simultaneously (Correct answer)
- The scheduling unit used to prioritize tensor operations
- A hardware pipeline stage for rasterization
Correct answer: A group of 32 threads that execute the same instruction simultaneously
A warp is the fundamental scheduling unit in NVIDIA GPUs, consisting of 32 threads that execute the same instruction in lockstep (SIMT).
Question 3: Which memory type on an NVIDIA GPU is shared among all threads within a thread block and has configurable size per SM?
- Global memory
- Constant memory
- Shared memory (Correct answer)
- Texture memory
Correct answer: Shared memory
Shared memory is an on-chip scratchpad memory partitioned per SM, accessible by all threads in a block with very low latency.
Question 4: What is 'warp divergence' and what is its performance impact?
- When warps from different blocks access the same cache line, causing contention
- When threads in a warp take different branches, causing serial execution of each branch (Correct answer)
- When a warp stalls waiting for a texture fetch to complete
- When two warps simultaneously write to the same register file
Correct answer: When threads in a warp take different branches, causing serial execution of each branch
Warp divergence occurs when threads in a warp follow different code paths (e.g., if/else), forcing the GPU to serialize execution of each path, reducing throughput.
Question 5: What is the function of the GigaThread Engine in NVIDIA GPU architecture?
- Execute tensor operations at high throughput using dedicated matrix units
- Distribute thread blocks across Streaming Multiprocessors (Correct answer)
- Manage NVLink communication between multiple GPUs
- Handle memory coalescing for global memory accesses
Correct answer: Distribute thread blocks across Streaming Multiprocessors
The GigaThread Engine is a global scheduler responsible for distributing thread blocks to available SMs across the GPU.
Question 6: Which NVIDIA GPU memory hierarchy level has the largest capacity but the highest access latency?
- L1/Shared memory
- L2 cache
- Register file
- HBM / GDDR global memory (Correct answer)
Correct answer: HBM / GDDR global memory
Global memory (HBM or GDDR) provides the largest capacity (tens of GBs) but incurs the highest latency (hundreds of cycles) compared to on-chip caches and registers.
Question 7: What distinguishes the Hopper architecture's Transformer Engine from earlier NVIDIA GPU generations?
- It adds dedicated ray-tracing BVH traversal units for each SM
- It introduces FP8 precision with hardware-managed scaling for transformer training (Correct answer)
- It replaces Tensor Cores with higher-clock scalar ALUs for sequential workloads
- It integrates NVSwitch directly onto the die for all-to-all GPU communication
Correct answer: It introduces FP8 precision with hardware-managed scaling for transformer training
The Hopper Transformer Engine introduces FP8 data formats with automatic scaling, enabling faster and more memory-efficient training of large transformer models.
NCA GPU Architecture & Fundamentals 2
What is the primary role of the L1 cache in NVIDIA GPU Streaming Multiprocessors (SMs)?