Classic CNN Architectures Flashcards

The VGGNet architecture is renowned for its simplicity and depth. What is the defining characteristic of its convolutional layers?

Answer: It exclusively uses very small 3x3 filters stacked on top of each other.

The key innovation and defining feature of the VGG architecture is its uniformity and simplicity, achieved by exclusively using small 3x3 convolutional filters stacked to increase the network's depth. This approach increases the effective receptive field and allows for learning more complex features with more non-linearities, while keeping the number of parameters manageable.

A key innovation of the GoogLeNet (Inception v1) architecture was the 'Inception module'. What is the primary purpose of this module?

Answer: To perform convolutions at multiple scales (1x1, 3x3, 5x5) in parallel and concatenate the results.

The core idea of the Inception module in GoogLeNet is to allow the network to capture features at multiple scales simultaneously. It achieves this by performing 1x1, 3x3, and 5x5 convolutions, along with a max pooling operation, in parallel within the same module. The outputs are then concatenated, creating a rich, multi-scale feature representation.

Which of the following classic CNN architectures was the first to successfully demonstrate that significantly deeper networks could be trained by using 'skip connections' or 'residual blocks' to address the vanishing gradient problem?

Answer: ResNet

ResNet (Residual Network) introduced the concept of residual blocks with skip connections. These connections create a shortcut path for the gradient to flow through, which helps mitigate the vanishing gradient problem that plagues very deep networks. This innovation allowed for the successful training of networks that were substantially deeper (e.g., 50, 101, or 152 layers) than previous architectures like AlexNet or VGG.

An engineer is choosing a classic CNN architecture for a task on hardware with limited computational resources. They need a model that is deep and accurate but has a significantly lower parameter count compared to VGG-16 or AlexNet. Which architecture would be the most suitable choice?

Answer: GoogLeNet (Inception v1)

GoogLeNet (Inception v1) was designed for computational efficiency. Through the heavy use of 1x1 convolutions for dimensionality reduction within its Inception modules, it achieved state-of-the-art accuracy with only about 6-7 million parameters, a fraction of AlexNet's (~60 million) and VGGNet's (~138 million) parameters.

Which of the following was a key innovation introduced in the AlexNet architecture that distinguished it from its predecessor, LeNet-5?

Answer: The use of the Rectified Linear Unit (ReLU) activation function.

While LeNet-5 used sigmoid or tanh activation functions, AlexNet was one of the first major architectures to popularize the use of the Rectified Linear Unit (ReLU). ReLU helped to significantly speed up the training process and mitigate the vanishing gradient problem, which was a major advantage for the deeper, 8-layer AlexNet architecture.

The LeNet-5 architecture, a pioneering CNN, was primarily designed for a specific task and had characteristic features for its time. Which of the following best describes LeNet-5?

Answer: A shallow network using average pooling and tanh/sigmoid activations, designed for handwritten digit recognition.

LeNet-5, developed by Yann LeCun, was a foundational 7-layer CNN designed for handwritten digit recognition on the MNIST dataset. Its architecture was characterized by its relatively shallow depth, alternating convolutional and average pooling layers, and the use of sigmoid or tanh activation functions.