# Class Imbalance

When one class vastly outnumbers others, so a model can score well by ignoring the minority entirely.

**Rests on:** Imbalanced Data, Two-Class and Multiclass Classifiers

*Machine Learning · Feature Engineering*


## Why learn this

You are working with data where one outcome is rare—perhaps only 1% of loan applications default, or 0.1% of patients have a specific disease. If you ignore this fact, your predictive model will simply guess the common outcome every time and claim 99.9% accuracy, which sounds impressive but is completely useless for finding the rare cases that actually matter. Understanding class imbalance prevents you from trusting misleading performance scores and gives you the tools to build models that actually detect those critical exceptions.


## The intuition

Imagine you are teaching a child to spot a rare bird. You show them a thousand pictures of pigeons and only ten of the rare bird. After a while, the child learns to say 'not the rare bird' to everything—and they are right 99% of the time. They have learned a rule that is almost always correct, but it is useless. A model does exactly the same thing. When one class vastly outnumbers another, the model can achieve a high accuracy score by always predicting the majority class. The model is not broken; it is doing exactly what it was trained to do—minimise the number of mistakes it makes. But a mistake that misses a rare cancer is not the same as a mistake that mislabels a healthy scan. The cost of errors is asymmetric, but accuracy treats every mistake as equally costly. So the first lesson of class imbalance is that accuracy becomes meaningless long before the model becomes useless. The second lesson is that you cannot just add more data—the imbalance is often a fact of the world, not a data collection error. The third is that you have a toolkit of responses: resample the data so the classes are more balanced, give the model different weights for different classes, and tune the threshold at which the model says 'yes'. Each of these changes what the model optimises, and each has a price. The goal is not to make the model perfectly balanced—it is to make it good at the task that actually matters.


## An analogy

Think of a security guard at an airport. They have to spot a smuggler in a stream of thousands of innocent travellers. If they stop everyone, they catch all smugglers but anger every passenger. If they stop no one, they miss all smugglers but keep the line moving. The guard's job is not to maximise the number of correct decisions in raw terms—it is to balance the cost of a false alarm (a delayed traveller, a lawsuit) against the cost of a missed smuggling attempt (a bomb on a plane). The guard has three tools: they can change how they look at the crowd (resampling—maybe they get more informants, or they ignore some obvious innocents), they can be more cautious by default (class weights—they raise their baseline suspicion), or they can change what they do with a borderline suspicion (threshold tuning—they only arrest when they are very sure, or they arrest on a hunch). The analogy breaks down when you remember that a model does not have judgment or fear—it has a single number that it compares to a threshold. The guard can weigh dozens of signals instantly; a model weighs what you give it, and if you give it only the crowd size and the number of suspicious glances, you have already limited it. Also, resampling is not free—you are discarding data or creating fake data, and that has its own risks. But the guard's core problem—minimising a cost that is not symmetrical—is exactly the model's problem.


## Definition

Class imbalance occurs when one outcome in your data is much less frequent than another, causing standard prediction methods to ignore the rare outcome because it is easier to always predict the common one.


## How it connects

This concept is central to Feature Engineering, which involves preparing and transforming raw data into a format that a machine learning model can use effectively. It also connects directly to Imbalanced Data, which is the broader category of problems where data distributions are skewed unevenly across any dimension, not just between classes in a prediction task.


## The mathematics

The mathematics behind class imbalance is the mathematics of counting and proportions. You have a set of examples, each with a label that puts it in one class. The imbalance is simply the ratio of the counts of the two classes. The core operation is the calculation of accuracy: the number of correct predictions divided by the total number of predictions. But you will learn that this single number hides the truth. The general branch of mathematics this draws on is arithmetic and proportions—you do not need calculus or algebra for the beginner stage. What you need is to count and divide.

The operation that matters is the confusion matrix: a four-celled table that counts how many examples of each actual class were predicted to be each class. From that table you extract ratios: precision (of everything the model said was the minority class, how much was correct), recall (of all the actual minority examples, how many the model caught), and the F1 score (a single number that balances precision and recall). You will work these by hand with real numbers before you see a formula. The final concept is the threshold: the model outputs a number, and you choose a cut-off to decide when it says 'positive'. Moving that threshold trades one kind of error for another.

Let us define the four counts using words first, then letters:

- TP, read 'T-P', means True Positives: examples where the actual class is positive and the model predicted positive.
- TN, read 'T-N', means True Negatives: examples where the actual class is negative and the model predicted negative.
- FP, read 'F-P', means False Positives: examples where the actual class is negative but the model incorrectly predicted positive.
- FN, read 'F-N', means False Negatives: examples where the actual class is positive but the model incorrectly predicted negative.

Then we calculate the metrics:

- Accuracy equals (TP plus TN) divided by (TP plus TN plus FP plus FN). This is the total number of correct predictions divided by the total number of examples.
- Precision equals TP divided by (TP plus FP). This is the number of correct positive predictions divided by the total number of times the model predicted positive.
- Recall equals TP divided by (TP plus FN). This is the number of correct positive predictions divided by the total number of actual positive examples.
- F1 score equals 2 multiplied by (Precision multiplied by Recall) divided by (Precision plus Recall). This formula combines precision and recall into one number. It is the harmonic mean, which penalizes extreme differences between the two.
- Baseline accuracy for always predicting majority equals (majority count) divided by (total count). This tells you what accuracy you get if you do nothing but guess the most common class every time.


## A worked example

We have a medical test for a rare disease. In our dataset, there are 100 patients: 90 are healthy (the majority class, class 0), and 10 are sick (the minority class, class 1). We train a model that always predicts 'healthy'. We calculate accuracy, and then we see the confusion matrix to reveal the problem.

Step 1: Count the predictions.
The model says 'healthy' for all 100 patients.
- TP (True Positives): The model predicted sick, and they were sick. Since it never predicted sick, this count is 0.
- TN (True Negatives): The model predicted healthy, and they were healthy. There are 90 healthy patients, so this count is 90.
- FP (False Positives): The model predicted sick, but they were healthy. Since it never predicted sick, this count is 0.
- FN (False Negatives): The model predicted healthy, but they were sick. There are 10 sick patients, and the model called them all healthy, so this count is 10.

Step 2: Calculate Accuracy.
Accuracy = (TP + TN) / (TP + TN + FP + FN)
Accuracy = (0 + 90) / (0 + 90 + 0 + 10)
Accuracy = 90 / 100
Accuracy = 0.90 or 90%.
The model is correct 90% of the time, which sounds good.

Step 3: Calculate Recall.
Recall measures how well the model finds the minority class (the sick patients).
Recall = TP / (TP + FN)
Recall = 0 / (0 + 10)
Recall = 0 / 10
Recall = 0.
The model catches zero percent of the sick patients. It is useless for finding the disease.

Step 4: Calculate Precision.
Precision measures how reliable the model's positive predictions are.
Precision = TP / (TP + FP)
Precision = 0 / (0 + 0)
Precision = 0 / 0.
Division by zero is undefined. Because the model never predicted 'sick', we cannot calculate precision. It is a mathematical gap caused by having no positive predictions to evaluate.

Result: Accuracy is 90%, but the model catches zero sick patients. This is why accuracy is meaningless under class imbalance—it only tells you about the majority class and ignores the failure to detect the minority class entirely.


## From scratch, in Python

```python
# Simulate a dataset with class imbalance
# We have 100 patients, 90 healthy and 10 sick
sick = 10
healthy = 90

# A model that always predicts 'healthy'
predictions = ['healthy'] * (sick + healthy)
true_labels = ['sick'] * sick + ['healthy'] * healthy

# Calculate accuracy
correct = sum(1 for p, t in zip(predictions, true_labels) if p == t)
accuracy = correct / len(true_labels)
print(f"Accuracy: {accuracy:.2f}")

# Build confusion matrix manually
tp = 0  # predicted sick and actually sick
tn = 0  # predicted healthy and actually healthy
fp = 0  # predicted sick but actually healthy
fn = 0  # predicted healthy but actually sick

for p, t in zip(predictions, true_labels):
    if p == t:
        if p == 'sick':
            tp += 1
        else:
            tn += 1
    else:
        if p == 'sick':
            fp += 1
        else:
            fn += 1

print(f"TP: {tp}, TN: {tn}, FP: {fp}, FN: {fn}")

# Precision: out of all predicted sick, how many are actually sick
# Recall: out of all actual sick, how many were caught
precision = tp / (tp + fp) if (tp + fp) > 0 else 0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0
f1 = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0

print(f"Precision: {precision:.2f}, Recall: {recall:.2f}, F1: {f1:.2f}")

# Now apply class weights: make the model care more about the sick class
# We give each sick example a weight of 9, and each healthy example a weight of 1
sample_weight = [9] * sick + [1] * healthy

# Weighted accuracy: sum of weights of correct predictions / total weight
weighted_correct = sum(9 for i in range(sick) if predictions[i] == true_labels[i]) + \
                   sum(1 for i in range(sick, len(true_labels)) if predictions[i] == true_labels[i])
total_weight = 9 * sick + 1 * healthy
weighted_accuracy = weighted_correct / total_weight
print(f"Weighted accuracy: {weighted_accuracy:.2f}")
```


## The same thing, with a framework

```python
In a real library like scikit-learn, you can do this with `sklearn.metrics.classification_report` and `sklearn.utils.class_weight.compute_class_weight`. The `classification_report` computes precision, recall and F1 for you. `compute_class_weight` gives you a formula for balancing classes. With `class_weight='balanced'` in a model like logistic regression, the model automatically gives more importance to the minority class.
```


## What the code does

The code starts by creating a list of labels for the true condition of each patient. `true_labels` has 10 `'sick'` entries followed by 90 `'healthy'` entries. The `predictions` list is all `'healthy'` because our model always says healthy. The `zip` function pairs each prediction with its true label—for an AI system, this is like laying two rows of cards side by side and reading them in pairs. We count how many pairs are equal, and that gives the number correct. Then we build the confusion matrix by iterating through each pair with a `for` loop. `tp` increases when the prediction is `'sick'` and the truth is `'sick'`; `tn` increases when both say healthy; and so on. Precision is the ratio of true sick predictions to all sick predictions, but since the model never predicts sick, both are zero, so we guard against division by zero with an `if`. Recall is zero because the model misses all ten sick patients. F1 is also zero because precision and recall are both zero. The final part introduces the idea of weights: we create a `sample_weight` list where each sick example has weight 9 and each healthy has weight 1. Weighted accuracy weights each correct prediction by its sample weight, so a correct catch of a sick patient counts for 9, and a correct healthy prediction counts for 1. The `sum` function adds up all the weighted correct predictions, and we divide by the total weight. This shows that even with a perfect model that always says healthy, weighted accuracy is not 90%—it is much lower, because the cost of missing a sick patient is now high. This foreshadows class weights in real models.


## Common misconceptions

**Accuracy is still a meaningful metric if it is high enough** — My model gets 99% accuracy, so it must be excellent. The low recall is just because the minority class is rare.

In a highly imbalanced set, accuracy is dominated by the majority class. A model that always predicts 'majority' will have accuracy equal to the majority proportion. If the minority is 1%, a model that never catches it has 99% accuracy. The accuracy number is uninformative because it does not distinguish between catching the minority and ignoring it. In fact: You must evaluate a model on the metric that reflects the cost of errors. For imbalanced problems, use precision, recall, F1, or a custom cost function. High accuracy with low recall is a sign of a broken model.

**Balancing classes always makes the model better** — I should resample so the classes are 50-50, then my model will be perfect.

Over-sampling the minority can lead to overfitting, because you are duplicating examples. Under-sampling the majority discards useful information. The true distribution matters for real-world performance—a model trained on a balanced set may be too optimistic about the minority's prevalence. In fact: Balancing is a tool, not a goal. Balance only if it improves the metric you care about, and validate on a held-out set that reflects the true imbalance. Sometimes class weights or threshold tuning are better than resampling.

**Class weight is the same as resampling** — Setting class_weight='balanced' is the same thing as sampling the data equally.

Class weights change the loss function during training, making errors on the minority class more expensive. Resampling changes the data distribution itself. They are related in effect, but not identical: weights only affect training, while resampling affects everything, including model evaluation if done wrong. In fact: Both can improve the model, but they act at different stages. Use class weights when you want to keep all your data and only change the optimisation. Use resampling when you have too much data or you need to alter the training set for other reasons.

**The threshold is fixed at 0.5** — The model outputs a probability, and the default threshold of 0.5 is the correct one.

The default threshold assumes symmetric error costs. When the cost of a false negative is high, you lower the threshold to catch more positives, even at the cost of more false positives. The optimal threshold depends on the cost matrix and the model's precision-recall trade-off. In fact: The threshold is a free parameter you should tune to your problem. Use the precision-recall curve to see how precision and recall trade off, and choose the point that minimises your expected cost.


## In short

Class imbalance is when one class vastly outnumbers another, making accuracy meaningless. The confusion matrix (TP, TN, FP, FN) is the foundation for meaningful metrics: precision, recall, and F1. The standard responses are resampling (balancing the data), class weights (changing the loss), and threshold tuning (changing the decision boundary). Always use a held-out set that preserves the true class distribution, and use the metric that reflects the asymmetric cost of errors for your task.


## Where this is used

Class imbalance is everywhere in applied machine learning. In **fraud detection**, fraudulent transactions are usually under 1% of all transactions. A model that flags everything as legitimate gets 99% accuracy, but is useless. You must use recall and precision, and you often set the threshold low so you catch more fraud at the cost of more false alarms. In **medical diagnosis**, a rare disease may appear in 1 in 10,000 scans. You cannot accept a model that never flags the disease, so you use class weights to force the model to treat each positive case as much more important than each negative. In **large language models (LLMs)**, pretraining data is not class-imbalanced in the traditional sense, but token distributions are skewed—common words like 'the' dominate, and rare words are under-represented. This is why LLMs sometimes struggle with low-frequency vocabulary. Resampling techniques are used to balance token frequencies during training. In **small language models on phones**, the same principle applies, but you have less data to work with, so you may need to balance more aggressively. In **retrieval-augmented generation (RAG)**, the retrieval step often has imbalance between relevant and irrelevant documents—you have to tune the retrieval threshold to avoid flooding the LLM with irrelevant context. In **agentic systems**, a model that decides to call a tool may see the 'no tool' action as overwhelmingly common, so you need to weight the tool calls to avoid the agent never calling a tool when it should. In **recommendation systems**, most items are never clicked, so a model that predicts 'no interaction' for everything scores well—you need to use recall and weighted metrics. In **vision**, object detection datasets have many background images and few objects of interest, so you use class weights or focal loss to address imbalance. In **speech**, keyword spotting models are trained on mostly silence or unknown words, so you need class weights to make the model detect the trigger word. The developer tools: `sklearn.model_selection.train_test_split` has a `stratify` parameter that preserves the class proportion across train and test splits. `imbalanced-learn` provides SMOTE, RandomUnderSampler, and other resampling methods. `sklearn.metrics` has `f1_score`, `precision_recall_curve`, and `auc`. A developer writes `model = LogisticRegression(class_weight='balanced')` or pipes SMOTE into a pipeline. What breaks when done badly: if you resample without care, you can introduce data leakage or destroy the true distribution; if you tune the threshold on the training set, you will overestimate performance; if you ignore the imbalance, your model is a zombie that correctly predicts the majority all day but is worthless for the task.


---

Read this on the web: https://basavapurushottam.com/learn-ai/class-imbalance
