NCA Inference Optimization & TensorRT 4

Question 1

Which TensorRT feature allows different network segments to run at different precisions (e.g., some layers in INT8, others in FP16)?

Accepted Answer

Mixed-precision execution via per-layer precision constraints

Answer

TensorRT supports setting per-layer precision and data-type constraints, enabling a single engine to mix INT8, FP16, and FP32 layers.

Question 2

What is the recommended approach to converting a PyTorch model for TensorRT deployment?

Accepted Answer

Export to ONNX first, then parse with TensorRT's ONNX parser

Answer

The standard workflow is PyTorch → ONNX export → TensorRT ONNX parser, which preserves the computational graph in an interoperable format.

Question 3

How does TensorRT handle operations not supported by its ONNX parser during model import?

Accepted Answer

It raises an error and requires the user to implement a custom TensorRT plugin for that operation

Answer

Unsupported ONNX ops cause a parse failure; the developer must register a TensorRT plugin that implements the missing operation on the GPU.

Question 4

What is 'kernel auto-tuning' in TensorRT?

Accepted Answer

During engine build, TensorRT benchmarks multiple CUDA kernel implementations for each layer and selects the fastest for the target GPU

Answer

TensorRT's build phase benchmarks candidate kernel implementations for each layer on the actual target GPU and permanently selects the fastest one.

Question 5

Which scenario best justifies using TensorRT's 'strongly typed' mode rather than default precision selection?

Accepted Answer

When specific layers must remain in FP32 for numerical correctness and automatic precision selection would convert them

Answer

Strongly typed mode forces TensorRT to respect explicit per-layer type constraints, preventing automatic precision downcasting for numerically sensitive layers.

(NCA) NVIDIA-Certified Associate Practice Test

(NCA) NVIDIA-Certified Associate Practice Test

NCA Inference Optimization & TensorRT 4