TensorFlow Model Training and Optimization 1 β Questions and Answers
Question 1: Which optimizer uses adaptive learning rates based on first and second moment estimates?
- Adam (Correct answer)
- SGD
- RMSprop
- Adagrad
Correct answer: Adam
Adam (Adaptive Moment Estimation) maintains per-parameter learning rates adapted using estimates of first and second moments of gradients.
Question 2: What does the learning rate hyperparameter control in gradient descent?
- The step size taken toward the minimum of the loss function (Correct answer)
- The number of training epochs
- The batch size
- The number of layers
Correct answer: The step size taken toward the minimum of the loss function
The learning rate determines how large a step is taken in the direction of the negative gradient during each parameter update.
Question 3: What is gradient clipping used for in TensorFlow training?
- Preventing exploding gradients by capping gradient magnitude (Correct answer)
- Speeding up backpropagation
- Removing small gradients
- Scheduling learning rate decay
Correct answer: Preventing exploding gradients by capping gradient magnitude
Gradient clipping limits gradient values to a maximum norm or value, preventing the exploding gradient problem in deep or recurrent networks.
Question 4: Which TensorFlow callback reduces the learning rate when a metric plateaus?
- ReduceLROnPlateau (Correct answer)
- LearningRateScheduler
- EarlyStopping
- TensorBoard
Correct answer: ReduceLROnPlateau
ReduceLROnPlateau monitors a metric and reduces the learning rate by a factor when improvement stalls for a specified number of epochs.
Question 5: What is the vanishing gradient problem in deep neural networks?
- Gradients become extremely small during backpropagation, preventing weight updates in early layers (Correct answer)
- Gradients disappear when GPU memory is full
- Loss function outputs zero
- Model weights become NaN
Correct answer: Gradients become extremely small during backpropagation, preventing weight updates in early layers
Vanishing gradients occur when gradients shrink exponentially as they propagate backward through many layers, making early layers train very slowly.
Question 6: Which technique trains a model in multiple steps with increasing dataset sizes?
- Curriculum learning (Correct answer)
- Transfer learning
- Knowledge distillation
- Federated learning
Correct answer: Curriculum learning
Curriculum learning trains the model on easier examples first, gradually increasing difficulty to improve convergence.
TensorFlow Model Training and Optimization 1
Which optimizer uses adaptive learning rates based on first and second moment estimates?