NCA Inference Optimization & TensorRT 5

Question 1

What is 'sparsity' acceleration in TensorRT, and which GPU architecture first introduced hardware support for it?

Accepted Answer

Exploiting 2:4 structured sparsity in weight tensors for 2× speedup, first supported on Ampere GPUs

Answer

NVIDIA's 2:4 structured sparsity (2 zeros per 4 weights) is decoded in hardware by Ampere Tensor Cores, delivering up to 2× speedup for sparse models.

Question 2

When deploying TensorRT engines on NVIDIA Jetson edge devices, which runtime library is used to execute the engine without the full TensorRT SDK?

Accepted Answer

TensorRT Runtime (libnvinfer.so) with only the runtime components, no builder required

Answer

Only the TensorRT runtime library (libnvinfer.so) is needed on the deployment device; the builder SDK is only required on the machine that creates the engine.

Question 3

Which TensorRT calibration algorithm is most suitable when calibration data is plentiful and accuracy must be maximized?

Accepted Answer

IInt8EntropyCalibrator2

Answer

IInt8EntropyCalibrator2 uses KL-divergence minimization to find optimal INT8 scale factors and is NVIDIA's recommended default for most vision models.

Question 4

What does setting 'builder_config.set_flag(trt.BuilderFlag.REFIT)' enable for a TensorRT engine?

Accepted Answer

Updating model weights in the compiled engine without rebuilding it from scratch

Answer

The REFIT flag builds the engine so its weights can be updated later via an IRefitter, avoiding costly full rebuilds when only weights change.

Question 5

In TensorRT, what is the function of 'mark_output()' when building a network via the Network Definition API?

Accepted Answer

It designates a specific tensor as a network output so TensorRT includes it in the engine's binding list

Answer

mark_output() tells TensorRT which tensors should be accessible as outputs through the engine's bindings during inference.

(NCA) NVIDIA-Certified Associate Practice Test

(NCA) NVIDIA-Certified Associate Practice Test

NCA Inference Optimization & TensorRT 5