NCA Edge AI & Embedded Systems 4 — Questions and Answers
Question 1: In a multi-camera edge inference system using DeepStream, what is the role of a 'nvstreammux' element?
- It compresses video streams before sending them to the cloud
- It batches frames from multiple input sources into a single buffer for efficient GPU inference (Correct answer)
- It decodes H.264/H.265 video streams using the CPU
- It applies ISP corrections to raw camera sensor data
Correct answer: It batches frames from multiple input sources into a single buffer for efficient GPU inference
nvstreammux batches frames from multiple sources into a single NvBufSurface buffer, enabling the GPU to process multiple streams in one inference call.
Question 2: What does the 'DLA' (Deep Learning Accelerator) on Jetson AGX Orin provide compared to using the GPU alone?
- Higher peak TOPS than the GPU for all model types
- A dedicated, power-efficient fixed-function accelerator for inference that offloads the GPU for concurrent workloads (Correct answer)
- Full compatibility with all TensorRT layer types without any fallback to GPU
- Real-time retraining of models at the edge without cloud connectivity
Correct answer: A dedicated, power-efficient fixed-function accelerator for inference that offloads the GPU for concurrent workloads
The DLA is a lower-power fixed-function accelerator optimized for inference, freeing the GPU to handle layers unsupported by DLA or additional concurrent tasks.
Question 3: A retail edge AI system must detect shoplifting events locally and store video clips without sending PII to the cloud. Which architectural pattern best satisfies this requirement?
- Cloud-first inference with edge buffering
- On-device inference with local encrypted storage and metadata-only cloud sync (Correct answer)
- Batch upload of raw video to cloud for offline analysis
- Federated learning with raw video shared across store nodes
Correct answer: On-device inference with local encrypted storage and metadata-only cloud sync
Running inference locally and storing only encrypted clips with anonymized metadata keeps PII on-device, satisfying privacy regulations while still enabling analytics.
Question 4: Which format does NVIDIA recommend as the intermediate representation when exporting a PyTorch model for TensorRT optimization on edge devices?
- ONNX (Correct answer)
- PMML
- CoreML
- TorchScript only
Correct answer: ONNX
ONNX (Open Neural Network Exchange) is the standard intermediate format for converting PyTorch or TensorFlow models before importing into TensorRT.
Question 5: What is the significance of 'latency vs. throughput' trade-off when tuning a TensorRT deployment on Jetson for a real-time robotics application?
- Maximizing throughput always minimizes latency in embedded GPU systems
- A large batch size increases throughput but also increases per-frame latency, which may violate real-time control loop requirements (Correct answer)
- Latency is irrelevant for robotics because control loops are asynchronous
- TensorRT automatically selects the optimal batch size to satisfy both metrics
Correct answer: A large batch size increases throughput but also increases per-frame latency, which may violate real-time control loop requirements
Larger batches amortize GPU overhead and improve throughput, but each frame must wait for the batch to fill, increasing latency beyond what a real-time control loop can tolerate.
Question 6: Which NVIDIA SDK provides pre-built perception models (object detection, segmentation, pose estimation) specifically optimized for Jetson edge deployment?
- CUDA Toolkit
- NVIDIA Metropolis
- Jetson Inference (jetson-inference library) (Correct answer)
- NVIDIA RAPIDS
Correct answer: Jetson Inference (jetson-inference library)
The jetson-inference library provides pre-trained models and C++/Python APIs built on TensorRT, making it easy to deploy perception tasks on Jetson without starting from scratch.
Question 7: When performing over-the-air (OTA) updates on an embedded Jetson device, which practice is most critical for maintaining system reliability?
- Applying updates directly to the running root filesystem without rebooting
- Using an A/B partition scheme so the device can revert to the previous known-good image if an update fails (Correct answer)
- Disabling update verification to speed up the OTA process
- Downloading updates through an unencrypted HTTP channel to reduce latency
Correct answer: Using an A/B partition scheme so the device can revert to the previous known-good image if an update fails
An A/B (dual-partition) scheme ensures the device always has a fallback boot partition, preventing a failed update from bricking the device.
NCA Edge AI & Embedded Systems 4
In a multi-camera edge inference system using DeepStream, what is the role of a 'nvstreammux' element?