Deep Learning & Neural Networks Flashcards

What is the primary function of an activation function in a neural network?

Answer: To add non-linearity to the network

The primary function of an activation function in a neural network is to introduce non-linearity into the model. Without activation functions, a neural network would only be able to learn linear transformations, regardless of its depth, limiting its ability to model complex relationships. By adding non-linearity, activation functions enable the network to learn and approximate highly complex, non-linear functions present in real-world data.

Which of the following is a common activation function?

Answer: ReLU

ReLU (Rectified Linear Unit) is one of the most common and widely used activation functions in deep learning. It outputs the input directly if it is positive, otherwise, it outputs zero, making it computationally efficient. Sigmoid is also an activation function, while Mean Squared Error is a loss function and Dropout is a regularization technique.

What does the term 'overfitting' refer to in deep learning?

Answer: The model performs well on training data but poorly on unseen data

Overfitting occurs when a deep learning model learns the training data too well, including its noise and specific details, rather than the underlying general patterns. This results in excellent performance on the training dataset but a significant drop in performance when the model is exposed to new, unseen data. The model fails to generalize because it has essentially memorized the training examples instead of learning robust features.

What technique is used to reduce overfitting in neural networks?

Answer: Dropout

Dropout is a powerful regularization technique specifically designed to reduce overfitting in neural networks. During training, it randomly sets a fraction of neurons to zero at each update, effectively 'dropping out' these neurons. This prevents the network from relying too heavily on specific neurons and forces it to learn more robust and generalized features, improving its performance on unseen data.

Which neural network is best suited for sequential data?

Answer: Recurrent Neural Network

Recurrent Neural Networks (RNNs) are specifically designed to process sequential data, where the order and context of elements are crucial. Unlike feedforward networks, RNNs have internal memory that allows them to retain information from previous steps in a sequence, making them ideal for tasks like natural language processing, speech recognition, and time series analysis. Convolutional Neural Networks (CNNs) are typically used for spatial data like images.

What is the purpose of backpropagation in a neural network?

Answer: To propagate errors backward and update weights

Backpropagation is the fundamental algorithm used to train neural networks by efficiently calculating the gradients of the loss function with respect to the network's weights. It propagates the error backward from the output layer through the hidden layers. These calculated gradients are then used by an optimizer to adjust the weights and biases, iteratively minimizing the loss and improving the model's accuracy.

Which component adjusts the weights during training in neural networks?

Answer: Optimizer

The optimizer is the component in neural networks responsible for adjusting the weights and biases during the training process. It uses the gradients calculated by backpropagation to determine how to update these parameters in a way that minimizes the loss function. Common optimizers include Stochastic Gradient Descent (SGD), Adam, and RMSprop, each with different strategies for navigating the loss landscape.

What is a convolution in CNNs primarily used for?

Answer: To detect local patterns in data

In Convolutional Neural Networks (CNNs), a convolution operation is primarily used to detect local patterns or features within the input data, typically images. A small filter (kernel) slides across the input, performing element-wise multiplications and summing the results, which highlights specific patterns like edges, textures, or shapes. This process creates feature maps that capture hierarchical representations of the input.

Which type of learning does deep learning primarily fall under?

Answer: Supervised learning

Deep learning primarily falls under supervised learning, where models learn from large datasets that include both input data and corresponding correct output labels. For example, in image classification, a deep learning model learns to identify objects by being trained on images labeled with their respective categories. While deep learning can also be applied to unsupervised and reinforcement learning, its most widespread and successful applications are in supervised tasks.