NCA NVIDIA Networking & Interconnects 4 — Questions and Answers
Question 1: Which congestion control algorithm does NVIDIA recommend for RoCE deployments to handle AI/ML all-reduce traffic?
- ECN-based DCQCN (Correct answer)
- TCP CUBIC
- InfiniBand credit-based flow control
- Token bucket rate limiting
Correct answer: ECN-based DCQCN
DCQCN (Data Center Quantized Congestion Notification) uses ECN marks to proactively rate-limit RoCE senders before queue overflow occurs.
Question 2: What is the role of the Subnet Manager (SM) in an InfiniBand fabric?
- Assigns IP addresses to compute nodes
- Initializes the fabric, assigns LIDs, and computes routing tables (Correct answer)
- Monitors GPU utilization across nodes
- Manages storage volume allocation
Correct answer: Initializes the fabric, assigns LIDs, and computes routing tables
The Subnet Manager discovers InfiniBand topology, assigns Local Identifiers (LIDs), and programs routing tables in all switches.
Question 3: What does SR-IOV provide when enabled on NVIDIA ConnectX NICs in a virtualized environment?
- Software-defined networking overlay
- Direct hardware NIC access for VMs without hypervisor overhead (Correct answer)
- Encrypted VM-to-VM communication
- Automatic VLAN tagging for VMs
Correct answer: Direct hardware NIC access for VMs without hypervisor overhead
SR-IOV creates Virtual Functions (VFs) that give VMs direct hardware access to NIC resources, bypassing the hypervisor data path.
Question 4: In NVIDIA's BlueField DPU, what distinguishes it from a standard ConnectX SmartNIC?
- Higher port speeds
- Embedded Arm cores that run a full OS and offload host CPU networking tasks (Correct answer)
- Support for InfiniBand only
- Built-in GPU compute cores
Correct answer: Embedded Arm cores that run a full OS and offload host CPU networking tasks
BlueField DPUs include Arm processor cores capable of running a full Linux OS, enabling complete networking stack offload from the host CPU.
Question 5: What is the maximum NVLink bandwidth available between two H100 GPUs in the same SXM server?
- 200 GB/s bidirectional
- 600 GB/s bidirectional
- 900 GB/s bidirectional (Correct answer)
- 1.2 TB/s bidirectional
Correct answer: 900 GB/s bidirectional
H100 NVLink provides 900 GB/s total bidirectional bandwidth per GPU (18 NVLink 4.0 lanes × 50 GB/s each).
Question 6: Which feature of NVIDIA's Quantum InfiniBand switches enables adaptive routing to avoid congested paths?
- Static ECMP routing
- Credit-based Dynamic Routing (credit-based flow control)
- Adaptive Routing with per-packet load balancing (Correct answer)
- OSPF-based dynamic routing
Correct answer: Adaptive Routing with per-packet load balancing
NVIDIA Quantum switches support adaptive routing that monitors queue depths and reroutes packets to less congested paths in real time.
Question 7: When deploying a multi-rail network for AI training, what problem does having multiple NICs per server solve?
- Provides redundancy only
- Increases aggregate bandwidth by spreading collective communication across multiple network paths (Correct answer)
- Reduces switch port count
- Enables GPU-to-CPU direct memory access
Correct answer: Increases aggregate bandwidth by spreading collective communication across multiple network paths
Multi-rail configurations assign multiple NICs to one server so collective communications use all available network bandwidth simultaneously.
NCA NVIDIA Networking & Interconnects 4
Which congestion control algorithm does NVIDIA recommend for RoCE deployments to handle AI/ML all-reduce traffic?