NCA GPU Architecture & Fundamentals 5 — Questions and Answers
Question 1: What is the function of NVIDIA's RT Cores in Turing and later architectures?
- Accelerate ray-box and ray-triangle intersection tests for real-time ray tracing (Correct answer)
- Execute recurrent neural network operations with sparse weight support
- Perform lossless compression of render targets stored in GPU memory
- Decode H.265 and AV1 video streams in hardware for display pipelines
Correct answer: Accelerate ray-box and ray-triangle intersection tests for real-time ray tracing
RT Cores are dedicated fixed-function hardware units that accelerate bounding volume hierarchy (BVH) traversal and ray-triangle intersection, the core bottleneck in real-time ray tracing.
Question 2: What is the difference between NVIDIA's 'compute capability' version and a GPU's CUDA toolkit version?
- Compute capability describes hardware feature support (SM architecture); CUDA toolkit version describes the software SDK release (Correct answer)
- Compute capability specifies the maximum GPU clock speed; CUDA toolkit version defines the memory bandwidth tier
- They are synonymous terms used interchangeably in NVIDIA documentation
- Compute capability refers to the number of SMs; CUDA toolkit version refers to the number of CUDA cores per SM
Correct answer: Compute capability describes hardware feature support (SM architecture); CUDA toolkit version describes the software SDK release
Compute capability (e.g., 8.0 for A100, 9.0 for H100) encodes the hardware SM feature set, while the CUDA toolkit is the software release that may support one or more compute capability levels.
Question 3: How does NVIDIA's Ampere architecture's sparsity feature improve Tensor Core throughput?
- It doubles effective Tensor Core throughput by skipping zero-valued weights in 2:4 structured sparse matrices (Correct answer)
- It prunes redundant CUDA cores at runtime to reduce power for sparse workloads
- It compresses activations using run-length encoding before storing them to HBM
- It enables irregular sparsity patterns via hardware pointer tables in the register file
Correct answer: It doubles effective Tensor Core throughput by skipping zero-valued weights in 2:4 structured sparse matrices
Ampere's Tensor Cores support 2:4 structured sparsity, where exactly 2 of every 4 weights are zero, allowing the hardware to skip zero multiplications and double MMA throughput for sparse models.
Question 4: What is the purpose of NVIDIA's GPUDirect RDMA technology?
- Allow GPU memory to be accessed directly by network adapters without CPU involvement, reducing latency (Correct answer)
- Enable direct GPU-to-SSD transfers using NVMe over PCIe without staging through DRAM
- Allow multiple GPU kernels to share the same CUDA context for reduced context-switch overhead
- Provide direct hardware access for peer GPUs to each other's L2 caches via PCIe
Correct answer: Allow GPU memory to be accessed directly by network adapters without CPU involvement, reducing latency
GPUDirect RDMA allows network adapters (InfiniBand or RoCE) to read and write GPU memory directly via PCIe, bypassing CPU and system memory and reducing communication latency for distributed training.
Question 5: In the context of NVIDIA GPU execution, what is a 'CUDA stream' and how does it enable performance optimization?
- A dedicated NVLink channel between two GPUs for overlapping communication and computation
- A sequence of CUDA operations that execute in order, allowing overlap of independent streams for concurrency (Correct answer)
- A hardware queue inside each SM that prefetches the next kernel's thread blocks
- A shared memory buffer that pipelines data between producer and consumer kernels
Correct answer: A sequence of CUDA operations that execute in order, allowing overlap of independent streams for concurrency
A CUDA stream is an ordered queue of GPU operations; operations in different streams can overlap (kernel execution, memory copies), enabling better GPU utilization through concurrency.
Question 6: What architectural characteristic primarily distinguishes a GPU from a CPU for parallel workloads?
- GPUs have a much larger last-level cache that hides DRAM latency for random access patterns
- GPUs have thousands of smaller, simpler cores optimized for throughput rather than a few complex latency-optimized cores (Correct answer)
- GPUs run at significantly higher clock frequencies than CPUs to compensate for their narrower SIMD units
- GPUs contain dedicated out-of-order execution engines in each CUDA core for instruction-level parallelism
Correct answer: GPUs have thousands of smaller, simpler cores optimized for throughput rather than a few complex latency-optimized cores
GPUs are throughput-oriented processors with thousands of simpler cores designed for massive parallelism, whereas CPUs have fewer, more complex cores optimized for low-latency sequential execution.
Question 7: What is CUDA Unified Memory and what does it simplify for GPU programmers?
- A shared L3 cache between CPU and GPU on NVLink-connected systems that eliminates DRAM accesses
- A single managed memory address space accessible by both CPU and GPU, with automatic data migration (Correct answer)
- A pooled register file shared across all SMs to eliminate register spilling under heavy occupancy
- A virtual memory layer that maps GPU global memory to CPU virtual addresses using GPU-assisted page tables
Correct answer: A single managed memory address space accessible by both CPU and GPU, with automatic data migration
Unified Memory creates a single coherent address space accessible by both CPU and GPU, with the CUDA runtime automatically migrating pages on demand, simplifying data management and eliminating explicit transfers.
NCA GPU Architecture & Fundamentals 5
What is the function of NVIDIA's RT Cores in Turing and later architectures?