NCA AI & Deep Learning Frameworks 4

Question 1

What problem does gradient clipping solve during training of deep neural networks, particularly RNNs?

Accepted Answer

Exploding gradients that cause parameter updates to become excessively large

Answer

Gradient clipping caps the norm or value of gradients before the optimizer step, preventing exploding gradients from destabilizing training.

Question 2

Which NVIDIA GPU architecture first introduced Tensor Cores for accelerating matrix multiply-accumulate operations?

Accepted Answer

Volta

Answer

NVIDIA Volta (V100) introduced the first generation of Tensor Cores, providing hardware acceleration for mixed-precision matrix operations used in deep learning.

Question 3

In ONNX (Open Neural Network Exchange), what is the primary purpose of the format?

Accepted Answer

Enabling model interoperability across different deep learning frameworks

Answer

ONNX defines a common format for representing neural network models, allowing models trained in one framework to be deployed in another.

Question 4

What is 'knowledge distillation' in the context of deep learning model compression?

Accepted Answer

Training a smaller 'student' model to mimic the outputs of a larger 'teacher' model

Answer

Knowledge distillation transfers knowledge from a large teacher model to a compact student model by training the student on the teacher's soft output probabilities.

Question 5

Which parallelism strategy in distributed training splits individual layers across multiple GPUs, with each GPU holding a subset of each layer's parameters?

Accepted Answer

Tensor parallelism

Answer

Tensor parallelism (intra-layer model parallelism) partitions weight matrices across GPUs, with each GPU computing a portion of each layer's operation.

(NCA) NVIDIA-Certified Associate Practice Test

(NCA) NVIDIA-Certified Associate Practice Test

NCA AI & Deep Learning Frameworks 4