NCA Inference Optimization & TensorRT 2

Question 1

Which TensorRT API call finalizes the network definition and produces an optimized inference engine?

Accepted Answer

builder.build_cuda_engine(network)

Answer

builder.build_cuda_engine(network) compiles and optimizes the network into a TensorRT engine ready for inference.

Question 2

What does TensorRT's 'workspace size' configuration control during the engine build process?

Accepted Answer

Maximum GPU memory the optimizer may use for algorithm selection

Answer

Workspace size limits the scratch GPU memory TensorRT can allocate when evaluating candidate layer implementations during optimization.

Question 3

In TensorRT, what is the purpose of the IOptimizationProfile when using dynamic shapes?

Accepted Answer

It defines min, opt, and max dimensions so the engine can be tuned across a range of input shapes

Answer

An IOptimizationProfile provides the min/opt/max shape bounds that TensorRT uses to auto-tune kernels for dynamic-shape networks.

Question 4

Which TensorRT precision mode typically offers the best throughput on Ampere and later GPUs while maintaining acceptable accuracy for most vision tasks?

Accepted Answer

FP16

Answer

FP16 leverages Tensor Core acceleration on Ampere GPUs and usually provides near-FP32 accuracy with roughly 2× throughput improvement.

Question 5

What is the role of a calibration dataset in TensorRT INT8 quantization?

Accepted Answer

It is used to compute activation range statistics that determine the optimal quantization scale factors

Answer

INT8 calibration runs representative data through the network to collect per-layer activation distributions used to derive scale factors.

(NCA) NVIDIA-Certified Associate Practice Test

(NCA) NVIDIA-Certified Associate Practice Test

NCA Inference Optimization & TensorRT 2