NCA Computer Vision & NLP Workloads 5 — Questions and Answers
Question 1: What is the purpose of 'streaming' mode in NVIDIA Triton Inference Server when handling NLP text generation?
- To distribute tokens across multiple GPUs
- To return generated tokens incrementally as they are produced rather than waiting for full completion (Correct answer)
- To cache key-value attention states to disk
- To quantize model weights during inference
Correct answer: To return generated tokens incrementally as they are produced rather than waiting for full completion
Streaming mode lets Triton send partial generation results token-by-token to the client, reducing perceived latency in interactive NLP applications.
Question 2: Which data format is the standard interchange format for exporting trained PyTorch or TensorFlow CV models before TensorRT optimization?
- Pickle (.pkl)
- ONNX (.onnx) (Correct answer)
- HDF5 (.h5)
- SafeTensors (.safetensors)
Correct answer: ONNX (.onnx)
ONNX (Open Neural Network Exchange) provides a framework-agnostic model representation that TensorRT can parse and optimize into a high-performance engine.
Question 3: In a multi-node NLP training job using NVIDIA NeMo and Slurm, which communication backend handles gradient synchronization across nodes?
- NCCL (NVIDIA Collective Communications Library) (Correct answer)
- OpenMPI only
- gRPC
- ZeroMQ
Correct answer: NCCL (NVIDIA Collective Communications Library)
NCCL provides highly optimized collective operations (AllReduce, AllGather, etc.) over NVLink and InfiniBand for multi-GPU and multi-node gradient synchronization.
Question 4: For a real-time NLP sentiment analysis service, what TensorRT optimization would you apply first to reduce FP32 model latency without retraining?
- INT4 weight-only quantization
- FP16 (half-precision) conversion (Correct answer)
- Sparse weight pruning
- Layer normalization fusion only
Correct answer: FP16 (half-precision) conversion
FP16 conversion halves the memory bandwidth requirement and leverages Tensor Core acceleration with minimal accuracy degradation and no calibration data required.
Question 5: Which NVIDIA NGC resource provides pretrained text embedding models that can be integrated directly into a semantic search or RAG application?
- CUDA Toolkit container
- NeMo Retriever embedding models (Correct answer)
- DeepStream reference graphs
- TAO Toolkit detection models
Correct answer: NeMo Retriever embedding models
NeMo Retriever provides GPU-optimized text embedding models on NGC designed for semantic search, vector database population, and RAG pipeline integration.
Question 6: What is 'KV cache' in the context of autoregressive NLP model inference, and why does it matter for GPU memory management?
- A disk cache for tokenized datasets that reduces preprocessing time
- Cached key and value attention tensors from previous tokens that avoid recomputation, consuming significant GPU memory (Correct answer)
- A Redis cache for model endpoint responses
- A key-value store for model hyperparameters
Correct answer: Cached key and value attention tensors from previous tokens that avoid recomputation, consuming significant GPU memory
During autoregressive decoding, KV cache stores past attention keys and values to avoid recomputing them, trading GPU memory for dramatically faster token generation.
Question 7: Which CUDA programming concept allows overlapping GPU compute kernels with host-to-device memory transfers to hide data loading latency in CV training?
- CUDA unified memory
- CUDA streams (Correct answer)
- CUDA cooperative groups
- CUDA dynamic parallelism
Correct answer: CUDA streams
CUDA streams enable concurrent execution of memory transfers and compute kernels on the GPU, allowing the next batch to be prefetched while the current batch is being processed.
NCA Computer Vision & NLP Workloads 5
What is the purpose of 'streaming' mode in NVIDIA Triton Inference Server when handling NLP text generation?