Deep Learning (Data Scientist) Practice Test — Questions and Answers

Question 1: The human brain is thought to have inspired a neural network model. The neural network is made up of many different components. Each neuron receives an input, processes it, and then outputs. Which of the following statements represents a genuine neuron correctly?

A neuron has several inputs but only one output.
A neuron has only one input and only one output.
Multiple inputs and outputs are found in a neuron.
Although a neuron has only one input, it has several outputs.
All of the following statements are correct. (Correct answer)

Correct answer: All of the following statements are correct.

The human brain contains a vast diversity of neuron types, each with unique structures and functions. While a typical neuron receives multiple inputs (dendrites) and its single axon can branch to provide multiple outputs to other neurons, some specialized neurons or simplified models might emphasize different aspects of connectivity. Therefore, depending on the specific neuron type or the level of abstraction, various statements about input/output configurations could be considered correct in a broad context.

Question 2: The most critical stage in constructing a neural network is determining each neuron's weight and bias. You can estimate any function if you can figure out how to get the right weight and bias for each neuron. What is the best strategy for dealing with this?

Find the best value by combining as many weights and biases as possible.
Iteratively check how far you are from the optimal values after assigning a value, then make little changes to the assigned values to improve them. (Correct answer)
Assign random values and pray to God that they are accurate.
None of the above

Correct answer: Iteratively check how far you are from the optimal values after assigning a value, then make little changes to the assigned values to improve them.

The most effective strategy for training a neural network involves iterative optimization. This means starting with initial weight and bias values, evaluating the network's performance (how far it is from optimal), and then making small, calculated adjustments to these values to reduce the error. This process, often guided by algorithms like gradient descent, is repeated until the network converges to a state where it performs well on the given task.

Question 3: What are the steps involved in employing a gradient descent algorithm? 1. Reiterate until you find the best weights of network 2. Go to each neurons which contributes to the error and change its respective values to reduce the error 3. Calculate error between the actual value and the predicted value 4.Pass an input through the network and get values from output layer 5. Initialize random weight and bias

5, 4, 3, 2, 1 (Correct answer)
1, 2, 3, 4, 5
4, 3, 1, 5, 2
3, 2, 1, 5, 4

Correct answer: 5, 4, 3, 2, 1

The gradient descent algorithm starts by initializing random weights and biases (5). An input is then passed through the network to generate a prediction (4), and the error between this prediction and the actual value is calculated (3). Based on this error, the algorithm determines how to adjust the weights and biases of each neuron to reduce the error (2). This entire sequence is repeated iteratively until the network's weights are optimized and the error is minimized (1).

Question 4: "Convolutional Neural Networks can change an input in a variety of ways (rotations or scaling)." True or False: Is the statement correct?

A) False (Correct answer)
B) True

Correct answer: A) False

This statement is false. Convolutional Neural Networks (CNNs) do not actively change an input's rotation or scaling. Instead, CNNs are designed to exhibit a degree of *invariance* to these transformations, meaning they can recognize patterns even if they are slightly rotated or scaled. This robustness is achieved through their architecture, including convolutional filters and pooling layers, which help them learn features that are less sensitive to minor variations in input presentation.

Question 5: Which of the following strategies is similar to dropout in a neural network in terms of operations?

Boosting
Stacking
Bagging (Correct answer)
None of the above

Correct answer: Bagging

Dropout is a regularization technique that randomly deactivates a subset of neurons during each training iteration. This prevents the network from relying too heavily on specific neurons and forces it to learn more robust features. This process is analogous to Bagging (Bootstrap Aggregating) in ensemble methods, where multiple models are trained on different subsets of data (or with different subsets of features/neurons) and their predictions are combined, improving generalization.

Question 6: A neural network's non-linearity is caused by which of the following?

Rectified Linear Unit (Correct answer)
Convolution function
Stochastic Gradient Descent
None of these

Correct answer: Rectified Linear Unit

The non-linearity in a neural network is introduced by activation functions, such as the Rectified Linear Unit (ReLU). Without non-linear activation functions, a multi-layer neural network would simply be a series of linear transformations, equivalent to a single linear layer, regardless of its depth. Non-linearity is crucial because it allows neural networks to learn and approximate complex, non-linear relationships present in real-world data, enabling them to solve sophisticated problems.

Question 7: Which of the following statements concerning model capacity (the ability of a neural network to approximate complex functions) is correct?

As learning rate increases, model capacity increases
As number of hidden layers increase, model capacity increases (Correct answer)
As dropout ratio increases, model capacity increases
None of the above

Correct answer: As number of hidden layers increase, model capacity increases

Model capacity refers to a neural network's ability to learn and represent complex functions. Increasing the number of hidden layers generally enhances the network's capacity, allowing it to model more intricate relationships and learn hierarchical features from the data. A deeper network can approximate a wider range of functions, provided it is trained effectively and doesn't overfit to the training data.

Question 8: The classification error of test data always lowers as the number of hidden layers in a Multi Layer Perceptron increases. Is this statement true or false?

A) False (Correct answer)
B) True

Correct answer: A) False

This statement is false. While increasing hidden layers can initially improve a Multi-Layer Perceptron's performance by allowing it to learn more complex patterns, it does not guarantee a continuous decrease in test classification error. Beyond a certain point, adding too many layers can lead to overfitting, where the model performs well on training data but poorly on unseen test data. It can also introduce training difficulties like vanishing gradients.

Question 9: In a perceptron, what is the sequence of the following tasks? 1. For a sample input, compute an output 2. If the prediction does not match the output, change the weights 3. Initialize weights of perceptron randomly 4.Go to the next batch of dataset

3, 1, 2, 4 (Correct answer)
1, 2, 3, 4
1, 4, 3, 2
4, 3, 2, 1

Correct answer: 3, 1, 2, 4

The training process for a perceptron begins by initializing its weights randomly (3). Then, for a given input, the perceptron computes an output (1). If this prediction does not match the desired output, the perceptron's weights are adjusted based on the error (2). This cycle is then repeated for the next batch or sample in the dataset (4), continuing until the perceptron learns to classify inputs correctly.

Question 10: Assume you have to change the settings to minimize the cost function. Which of the following methods could be utilized in this situation?

Bayesian Optimization
Exhaustive Search
Random Search
Any of these (Correct answer)

Correct answer: Any of these

To minimize a cost function by adjusting hyperparameters, various optimization methods can be employed. Random Search explores the hyperparameter space randomly, while Exhaustive Search (or Grid Search) systematically checks all predefined combinations. Bayesian Optimization uses a probabilistic model to intelligently guide the search, making it more efficient. All these techniques are valid approaches for finding optimal settings that minimize the cost function.

Question 11: Weight sharing occurs in which neural net architecture?

Fully Connected Neural Network
Convolutional neural Network
Recurrent Neural Network
Both B and C (Correct answer)

Correct answer: Both B and C

Weight sharing is a fundamental characteristic of both Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). In CNNs, the same convolutional filters (weights) are applied across different spatial locations of an input, enabling feature detection regardless of position. In RNNs, the same set of weights is reused for each time step in a sequence, allowing the network to learn temporal dependencies and process sequences of varying lengths efficiently.

Question 12: Normalization by batch is helpful because

It is a very efficient backpropagation technique
It normalizes (changes) all the input before sending it to the next layer (Correct answer)
It returns back the normalized mean and standard deviation of weights
None of the above

Correct answer: It normalizes (changes) all the input before sending it to the next layer

Batch Normalization is beneficial because it normalizes the inputs to each layer within a mini-batch during training. This process scales and shifts the activations to have a mean of zero and a standard deviation of one. By doing so, it stabilizes and accelerates the training process, allows for higher learning rates, and can also act as a form of regularization, improving the network's overall performance and generalization.

Question 13: We specify a metric called bayes error, which is the error we expect to attain, instead of trying to reach absolute zero error. What is the rationale for employing Bayes error?

Limited training data
Input variables may not contain complete information about the output variable
System (that creates input-output mapping) may be stochastic
All the above (Correct answer)

Correct answer: All the above

Bayes error represents the theoretical minimum error rate for a given problem, which no model can surpass. We use it as a benchmark because real-world data often contains inherent noise, ambiguity, or incomplete information due to limited training data, input variables not fully capturing the output, or the underlying system being stochastic. Therefore, aiming for absolute zero error is often unrealistic, and Bayes error provides a practical lower bound for model performance.

Question 14: Which of the following strategies is used to cope with overfitting in a neural network?

Regularization
Batch Normalization
Dropout
All of the above (Correct answer)

Correct answer: All of the above

Overfitting occurs when a neural network learns the training data too well, including noise, leading to poor performance on unseen data. Regularization techniques (like L1/L2 penalties) discourage large weights, Dropout randomly deactivates neurons during training to prevent over-reliance on specific features, and Batch Normalization helps stabilize training and can also have a regularization effect. All these strategies are commonly used to mitigate overfitting and improve a model's ability to generalize.

Question 15: In a supervised learning task, the number of neurons in the output layer should match the number of classes (where the number of classes is larger than 2). Is this statement true or false?

A) False (Correct answer)
B) True

Correct answer: A) False

This statement is false. While for multi-class classification (more than two classes) using a softmax activation, the number of output neurons typically matches the number of classes, this is not universally true. For binary classification (two classes), a single output neuron with a sigmoid activation function is often sufficient. This neuron outputs the probability of one class, with the other class's probability being 1 minus that value.

Question 16: Y = ax^2 + bx + c (polynomial equation of degree 2) Is it possible to represent this equation using a neural network with a single hidden layer and a linear threshold?

A) No (Correct answer)
B) Yes

Correct answer: A) No

No, it is not possible to represent a non-linear polynomial equation like Y = ax^2 + bx + c using a neural network with a single hidden layer and a *linear* threshold (or activation function). Linear activation functions, regardless of the number of layers, can only model linear relationships. To represent non-linear functions, a neural network requires at least one hidden layer combined with a non-linear activation function (e.g., ReLU, sigmoid, tanh).

Question 17: Which of the following statements describes early stopping the best?

Simulate the network on a test dataset after every epoch of training. Stop training when the generalization error starts to increase (Correct answer)
A faster version of backpropagation, such as the `Quickprop’ algorithm
Train the network until a local minimum in the error function is reached
Add a momentum term to the weight update in the Generalized Delta Rule, so that training converges more quickly

Correct answer: Simulate the network on a test dataset after every epoch of training. Stop training when the generalization error starts to increase

Early stopping is a regularization technique that prevents overfitting by monitoring the model's performance on a separate validation dataset during training. Training continues as long as the validation error decreases. When the validation error begins to increase, indicating that the model is starting to overfit the training data, training is halted, and the model weights from the epoch with the best validation performance are typically restored.

Question 18: In a neural network, what is a dead unit?

A unit which does not respond completely to any of the training patterns
The unit which produces the biggest sum-squared error
A unit which doesn’t update during training by any of its neighbour (Correct answer)
None of the above

Correct answer: A unit which doesn’t update during training by any of its neighbour

A 'dead unit' in a neural network, particularly common with ReLU activation functions, refers to a neuron whose output is always zero for all inputs. Consequently, its weights and biases never receive updates during backpropagation because the gradient through it is zero. Such a neuron effectively becomes inactive and stops contributing to the network's learning process or its output.

Question 19: What if we utilize an excessively high learning rate?

Can’t Say
Network will converge
Network will not converge (Correct answer)

Correct answer: Network will not converge

If an excessively high learning rate is used during neural network training, the optimization algorithm will take steps that are too large. Instead of gradually converging towards the minimum of the loss function, the weights might overshoot the minimum, causing the loss to oscillate wildly or even diverge. This prevents the network from effectively learning and converging to an optimal solution.

Question 20: Translation invariance is kept when a pooling layer is added to a convolutional neural network. Is this statement true or false?

A) False
B) True (Correct answer)

Correct answer: B) True

This statement is true. Pooling layers, such as max pooling or average pooling, are crucial components in Convolutional Neural Networks that contribute to translation invariance. By downsampling the feature maps, pooling makes the network less sensitive to the exact position of a feature within the input. If a feature shifts slightly, the pooling operation can still detect it in the same pooled region, enhancing the network's ability to recognize objects regardless of minor translations.

Question 21: Assume an ImageNet dataset is used to train a convolutional neural network (Object recognition dataset). The trained model is then given a fully white image as an input, with the output probabilities for all classes being equal. Is this statement true or false?

A) False (Correct answer)
B) True

Correct answer: A) False

This statement is false. A CNN trained on a diverse dataset like ImageNet learns complex features from real-world images. When presented with a completely white image, which is an out-of-distribution input, the network would likely produce an arbitrary, non-uniform distribution of probabilities across classes. It would not output equal probabilities for all classes, as the input does not resemble any learned pattern, and the network's internal biases would still influence the output.

Question 22: When the data is too large to handle in RAM at the same time, which gradient technique is more advantageous?

A) Stochastic Gradient Descent (Correct answer)
B) Full Batch Gradient Descent

Correct answer: A) Stochastic Gradient Descent

Stochastic Gradient Descent (SGD) is more advantageous when dealing with datasets too large to fit into RAM. Unlike Full Batch Gradient Descent, which computes gradients over the entire dataset, SGD updates model weights after processing each individual training example or a small mini-batch. This approach significantly reduces memory requirements, as it doesn't need to load the entire dataset into memory for each gradient calculation, making it scalable for large-scale data.

Question 23: Which architecture of neural network would be more suited to address an image identification challenge (recognizing a cat in a photo)?

Perceptron
Multi Layer Perceptron
Convolutional Neural Network (Correct answer)
Recurrent Neural network

Correct answer: Convolutional Neural Network

A Convolutional Neural Network (CNN) is the most suitable architecture for image identification challenges like recognizing a cat in a photo. CNNs are specifically designed to process image data, utilizing convolutional layers to automatically learn hierarchical spatial features such as edges, textures, and shapes. Their architecture, including pooling and weight sharing, makes them highly effective at recognizing patterns and objects within images, outperforming other network types for visual tasks.

Question 24: Consider the following scenario. There isn't a lot of data in the problem you're trying to address. You have a pre-trained neural network that was trained on a similar problem, which is fortunate. Which of the following approaches would you use to put this pre-trained network to work for you?

Freeze all the layers except the last, re-train the last layer (Correct answer)
Fine tune the last couple of layers only
Re-train the model for the new dataset
Assess on every layer how the model performs and only select a few of them

Correct answer: Freeze all the layers except the last, re-train the last layer

When dealing with limited data for a new problem but having a pre-trained network on a similar, larger dataset, the best approach is transfer learning by freezing most layers. Specifically, you would freeze all layers except the last one (or a few final layers) and retrain only those. This leverages the powerful, general feature representations learned by the pre-trained network in its earlier layers, while adapting the output to your specific new task with minimal new data.

Question 25: The performance of a convolutional network would always improve as the size of the convolutional kernel was increased.

A) False (Correct answer)
B) True

Correct answer: A) False

This statement is false. Increasing the size of the convolutional kernel does not always improve a CNN's performance. While larger kernels can capture broader spatial contexts, they also increase the number of parameters, computational cost, and the risk of overfitting, especially with smaller datasets. Smaller kernels (e.g., 3x3) are often preferred as they can capture fine-grained details and, when stacked, can achieve a large receptive field with fewer parameters, leading to more efficient and robust networks.

Deep Learning (Data Scientist) Practice Test

The human brain is thought to have inspired a neural network model.
The neural network is made up of many
different components.
Each neuron receives an input, processes it, and then outputs.
Which of the following statements represents a genuine neuron correctly?