NLP Text Classification 2 — Questions and Answers
Question 1: Which regularization technique is commonly applied in neural text classifiers to prevent overfitting by randomly deactivating neurons during training?
- Batch normalization
- L2 weight decay
- Dropout (Correct answer)
- Gradient clipping
Correct answer: Dropout
Dropout randomly sets a fraction of neuron activations to zero during each training pass, forcing the network to learn redundant representations and reducing overfitting.
Question 2: What is the key advantage of fine-tuning a pre-trained language model (e.g., BERT) for text classification compared to training from scratch?
- It requires more labeled data to achieve good performance
- It leverages general language knowledge learned from large corpora, needing far less task-specific data (Correct answer)
- It eliminates the need for any labeled training data
- It always produces shorter inference times than simpler models
Correct answer: It leverages general language knowledge learned from large corpora, needing far less task-specific data
Pre-trained models encode rich linguistic knowledge from massive unlabeled corpora, so fine-tuning requires only a small labeled dataset to adapt them to a specific classification task.
Question 3: In text classification, what problem does 'class imbalance' refer to?
- Having too many features relative to the number of samples
- One class having significantly more examples than others in the training set (Correct answer)
- Using different loss functions for different classes
- Misalignment between training and test vocabulary
Correct answer: One class having significantly more examples than others in the training set
Class imbalance occurs when training data has a disproportionate number of examples for some classes, causing the model to be biased toward predicting the majority class.
Question 4: Which technique involves creating synthetic minority-class examples to address class imbalance in text classification?
- Subsampling
- SMOTE (Synthetic Minority Over-sampling Technique) (Correct answer)
- Pruning
- Dimensionality reduction
Correct answer: SMOTE (Synthetic Minority Over-sampling Technique)
SMOTE generates new synthetic examples for the minority class by interpolating between existing minority-class instances, helping balance the training distribution.
Question 5: What is hierarchical text classification?
- Classifying text based on its reading difficulty level
- Organizing categories into a tree structure where documents are classified from broad to specific (Correct answer)
- Ranking documents by relevance rather than assigning a single category
- Using multiple layers of neural networks for classification
Correct answer: Organizing categories into a tree structure where documents are classified from broad to specific
Hierarchical classification leverages a taxonomy (e.g., Science → Biology → Genetics) to make classification decisions at multiple levels of specificity, improving accuracy on fine-grained categories.
Question 6: When using cross-validation for evaluating a text classifier, what does 'k-fold' mean?
- The dataset is split into k classes for stratified sampling
- The data is divided into k equal parts, each used once as the validation set while the rest train the model (Correct answer)
- The model is trained k times with different random seeds to test stability
- k hyperparameters are tuned simultaneously using grid search
Correct answer: The data is divided into k equal parts, each used once as the validation set while the rest train the model
In k-fold cross-validation, the dataset is split into k folds; the model trains on k-1 folds and validates on the remaining fold, rotating through all k folds to get a robust performance estimate.
Question 7: Which loss function is most commonly used when training a neural network for multi-class text classification?
- Mean Squared Error (MSE)
- Binary Cross-Entropy
- Categorical Cross-Entropy (Correct answer)
- Hinge Loss
Correct answer: Categorical Cross-Entropy
Categorical cross-entropy measures the divergence between the predicted probability distribution over all classes and the true one-hot encoded label, making it ideal for multi-class problems.
NLP Text Classification 2
Which regularization technique is commonly applied in neural text classifiers to prevent overfitting by randomly deactivating neurons during training?