CNN - Convolutional Neural Networks Classic CNN Architectures Questions and Answers — Questions and Answers
Question 1: The VGGNet architecture is renowned for its simplicity and depth. What is the defining characteristic of its convolutional layers?
- It uses large 11x11 and 5x5 filters in the initial layers.
- It exclusively uses very small 3x3 filters stacked on top of each other. (Correct answer)
- It introduces parallel 1x1, 3x3, and 5x5 filters in each block.
- It employs skip connections to bypass certain convolutional layers.
Correct answer: It exclusively uses very small 3x3 filters stacked on top of each other.
The key innovation and defining feature of the VGG architecture is its uniformity and simplicity, achieved by exclusively using small 3x3 convolutional filters stacked to increase the network's depth. This approach increases the effective receptive field and allows for learning more complex features with more non-linearities, while keeping the number of parameters manageable.
Question 2: A key innovation of the GoogLeNet (Inception v1) architecture was the 'Inception module'. What is the primary purpose of this module?
- To drastically increase the network depth by using residual connections.
- To simplify the architecture by using only 3x3 convolutions.
- To perform convolutions at multiple scales (1x1, 3x3, 5x5) in parallel and concatenate the results. (Correct answer)
- To replace all fully connected layers with global average pooling to reduce parameters.
Correct answer: To perform convolutions at multiple scales (1x1, 3x3, 5x5) in parallel and concatenate the results.
The core idea of the Inception module in GoogLeNet is to allow the network to capture features at multiple scales simultaneously. It achieves this by performing 1x1, 3x3, and 5x5 convolutions, along with a max pooling operation, in parallel within the same module. The outputs are then concatenated, creating a rich, multi-scale feature representation.
Question 3: Which of the following classic CNN architectures was the first to successfully demonstrate that significantly deeper networks could be trained by using 'skip connections' or 'residual blocks' to address the vanishing gradient problem?
- AlexNet
- VGG-16
- LeNet-5
- ResNet (Correct answer)
Correct answer: ResNet
ResNet (Residual Network) introduced the concept of residual blocks with skip connections. These connections create a shortcut path for the gradient to flow through, which helps mitigate the vanishing gradient problem that plagues very deep networks. This innovation allowed for the successful training of networks that were substantially deeper (e.g., 50, 101, or 152 layers) than previous architectures like AlexNet or VGG.
Question 4: An engineer is choosing a classic CNN architecture for a task on hardware with limited computational resources. They need a model that is deep and accurate but has a significantly lower parameter count compared to VGG-16 or AlexNet. Which architecture would be the most suitable choice?
- LeNet-5
- AlexNet
- GoogLeNet (Inception v1) (Correct answer)
- VGG-19
Correct answer: GoogLeNet (Inception v1)
GoogLeNet (Inception v1) was designed for computational efficiency. Through the heavy use of 1x1 convolutions for dimensionality reduction within its Inception modules, it achieved state-of-the-art accuracy with only about 6-7 million parameters, a fraction of AlexNet's (~60 million) and VGGNet's (~138 million) parameters.
Question 5: Which of the following was a key innovation introduced in the AlexNet architecture that distinguished it from its predecessor, LeNet-5?
- The use of average pooling for subsampling.
- The use of the Rectified Linear Unit (ReLU) activation function. (Correct answer)
- The introduction of the Inception module.
- A shallow architecture with only 5 layers.
Correct answer: The use of the Rectified Linear Unit (ReLU) activation function.
While LeNet-5 used sigmoid or tanh activation functions, AlexNet was one of the first major architectures to popularize the use of the Rectified Linear Unit (ReLU). ReLU helped to significantly speed up the training process and mitigate the vanishing gradient problem, which was a major advantage for the deeper, 8-layer AlexNet architecture.
Question 6: The LeNet-5 architecture, a pioneering CNN, was primarily designed for a specific task and had characteristic features for its time. Which of the following best describes LeNet-5?
- A deep 19-layer network using only 3x3 filters for large-scale color image classification.
- A shallow network using average pooling and tanh/sigmoid activations, designed for handwritten digit recognition. (Correct answer)
- A model that introduced residual connections to train networks over 100 layers deep.
- An architecture that used parallel multi-scale convolutions and global average pooling.
Correct answer: A shallow network using average pooling and tanh/sigmoid activations, designed for handwritten digit recognition.
LeNet-5, developed by Yann LeCun, was a foundational 7-layer CNN designed for handwritten digit recognition on the MNIST dataset. Its architecture was characterized by its relatively shallow depth, alternating convolutional and average pooling layers, and the use of sigmoid or tanh activation functions.
CNN - Convolutional Neural Networks Classic CNN Architectures Questions and Answers
The VGGNet architecture is renowned for its simplicity and depth.
What is the defining characteristic of its convolutional layers?