# Rule-Based Classifiers

Classifiers expressed as explicit if-then rules, either handcrafted or induced from data.

**Rests on:** Decision Trees, Two-Class and Multiclass Classifiers

*Machine Learning · Machine Learning Foundations*


## Why learn this

You are starting your journey into machine learning from scratch. Rule-based classifiers are the first and simplest kind of classifier you will meet, and they are the perfect entry point because they require no mathematics beyond arithmetic and no programming beyond basic Python. They are also the foundation for more advanced ideas: every decision tree is a set of rules, every random forest is a collection of trees, and even the most complex neural network can be inspected as a set of rules after training. Learning rule-based classifiers will give you the vocabulary and intuition you need to understand how all classifiers work: they all take input values, apply some computation, and produce a label. By the end of this page, you will be able to hand-build a simple classifier for a real problem, explain what 'training' means in this context, and recognise where rules are genuinely useful versus where they fail.


## The intuition

Imagine you are a doctor in a small clinic. A patient comes in with a fever, a cough, and a sore throat. You don't run any tests—you just look at the symptoms and decide: 'This is a cold' or 'This is the flu.' How do you do that? You have a set of rules in your head: if the fever is high and the cough is dry, it's probably the flu; if the fever is low and the cough is wet, it's probably a cold. You don't calculate anything mathematical; you just apply rules. That is exactly what a rule-based classifier does. It looks at the input values (the symptoms) and applies a series of if-then rules to decide which class (the disease) the input belongs to. The rules are explicit, which means a human can read them and understand exactly why the machine made its decision. 

Now, why do we call it a 'classifier'? Because its job is to classify—to put things into categories. Think of sorting mail: letters go in one pile, packages in another. You classify each item based on rules: if it's flat and rectangular, it's a letter; if it's bulky, it's a package. A rule-based classifier does the same thing with data. It might classify emails as spam or not-spam, or customers as likely to churn or not. 

The key idea is that the rules are explicit. They are written down in plain language, like 'IF the patient has a high fever AND a dry cough, THEN diagnose the flu.' This is different from many other machine learning methods, where the decision process is hidden inside complex numbers and equations. With rules, you can open the box and see exactly what's inside. That transparency is valuable in situations where you need to justify a decision, like a bank denying a loan or a hospital recommending a treatment.


## An analogy

Think of a passport control officer at an airport. The officer has a set of rules written in a manual: if the passport is valid, if the visa is correct, if the person's name matches, then allow entry; otherwise, deny entry. The officer does not compute a score or use any statistical model—they just follow the rules step by step. This is rule-based classification: a sequence of if-then checks that lead to a decision. 

The beauty of this analogy is that it shows how rules can be hierarchical. The officer might first check if the passport is from a visa-free country. If yes, they skip the visa check. If no, they check the visa. This is a tree structure, which is exactly how many rule-based classifiers are organised. Each rule is a branch, and the final decision is a leaf. 

Now, where does this analogy break down? A passport officer only has a handful of rules, and they can handle exceptions easily. In machine learning, rule-based classifiers can have thousands of rules, and they are often generated automatically from data, not written by hand. Also, a human officer can use common sense and context—if a rule leads to an absurd outcome, they can override it. A rule-based classifier cannot; it will follow its rules blindly, even if they are wrong. So while the analogy captures the essence of rules, it does not capture the scale, the automatic generation, or the inflexibility of a real system.


## Definition

A rule-based classifier is a model that assigns a class label to an input by applying a set of explicit if-then rules, where each rule matches the input's feature values to a condition and maps it to a decision.


## How it connects

You are starting from scratch, so you haven't mastered any other machine learning concepts yet. But this page is your first step. Rule-based classifiers are the simplest form of a classifier, and they will connect to everything you learn later. For example, decision trees (which you'll see in the 'Decision Trees' topic) are essentially rule-based classifiers with a specific structure. The rules you learn here will reappear there. Also, the idea of 'features' (the input values) and 'labels' (the output classes) is used in every classifier, from K-Nearest Neighbours to neural networks. So by mastering rule-based classifiers, you are building the foundation for all of machine learning.


## The mathematics

A rule-based classifier makes decisions by checking simple conditions against your data. It does not use complex formulas, algebra, or calculus. It uses only basic arithmetic (addition, subtraction, multiplication, division) and comparison (greater than, less than, equal to).

Think of a rule as a set of instructions that looks like this:
IF [condition is true] THEN [do this action]

The "condition" is just a check. For example, you might check if a number is bigger than another number. The "action" is usually assigning a label or category to the item being checked.

How a single rule works:
1. You have input values (features). Let's call one of them `feature_value`. This is just a number from your data, like an age, a price, or an income amount.
2. You have a threshold. This is a fixed number chosen by the person building the classifier. Let's call it `threshold_value`.
3. You compare `feature_value` to `threshold_value`. Common comparisons are:
   - Is `feature_value` greater than `threshold_value`? (written as >)
   - Is `feature_value` less than `threshold_value`? (written as <)
   - Is `feature_value` equal to `threshold_value`? (written as ==)

If the comparison is true, the condition for that part of the rule is satisfied.

Combining conditions:
Sometimes a rule needs more than one check. You can combine checks using the words AND or OR.
- AND means every single check must be true. If you have two checks, both must pass.
- OR means at least one of the checks must be true.

For example, a rule might say:
IF (income > 50) AND (debt < 20) THEN approve loan.
Here, you check if income is greater than 50. Then you check if debt is less than 20. Only if both are true does the rule result in "approve".

How a set of rules works:
A classifier usually has many rules. Let's call the whole list of rules `RuleSet`. Each individual rule in this list is called `rule_r`.
Each `rule_r` has two parts:
1. A condition, which we will call `Condition_r`. This is the IF part.
2. A label or outcome, which we will call `Label_r`. This is the THEN part.

When you give a new piece of data to the classifier, it goes through the rules in order (usually from top to bottom):
1. It takes the first rule, `rule_1`.
2. It checks if `Condition_1` is true for this data.
3. If `Condition_1` is true, the classifier stops looking at other rules and returns `Label_1` as the answer.
4. If `Condition_1` is false, it moves to the next rule, `rule_2`, and checks `Condition_2`.
5. It continues this process until it finds a rule whose condition is true.

What if no rule matches?
It is possible that none of the conditions in your list are true for the given data. In that case, the classifier uses a default label. This is like a "catch-all" answer. For example, if no specific rule applies, the default might be "reject" or "unknown".

Summary of symbols used:
- `feature_value`: A number from your input data (e.g., income).
- `threshold_value`: A fixed number used for comparison (e.g., 50,000).
- `RuleSet`: The entire list of rules in the classifier.
- `rule_r`: One specific rule in the list.
- `Condition_r`: The logical test part of `rule_r` (the IF part).
- `Label_r`: The result or category assigned by `rule_r` (the THEN part).


## A worked example

Let's walk through a simple example. Imagine a bank wants to decide whether to approve a loan application. They use one rule-based classifier with just one rule.

The Rule:
IF (monthly_income >= 30) AND (total_debt <= 10) THEN approve_loan
ELSE reject_loan

Here, the numbers are in thousands of rupees. So "30" means 30,000 rupees and "10" means 10,000 rupees.

The Applicant's Data:
- monthly_income = 45 (which is 45,000 rupees)
- total_debt = 8 (which is 8,000 rupees)

Step-by-step evaluation:

1. Check the first part of the condition: Is monthly_income >= 30?
   - We substitute the applicant's income: Is 45 >= 30?
   - 45 is greater than 30, so this part is TRUE.

2. Check the second part of the condition: Is total_debt <= 10?
   - We substitute the applicant's debt: Is 8 <= 10?
   - 8 is less than 10, so this part is TRUE.

3. Combine the results with AND:
   - The rule requires BOTH parts to be true because of the word "AND".
   - We have TRUE AND TRUE.
   - Therefore, the entire condition for this rule is TRUE.

4. Determine the outcome:
   - Since the condition is true, we follow the "THEN" part of the rule.
   - The label assigned is `approve_loan`.

Result: The classifier approves the loan for this applicant.

What if the data were different?
Suppose another applicant has:
- monthly_income = 25
- total_debt = 15

Step-by-step evaluation:
1. Check: Is 25 >= 30?
   - No, 25 is less than 30. This part is FALSE.
2. Because the first part is FALSE, and the rule uses AND, the entire condition is FALSE immediately. We do not even need to check the debt in many systems (this is called short-circuiting), but even if we did:
   - Check: Is 15 <= 10?
   - No, 15 is greater than 10. This part is also FALSE.
3. Since the condition is FALSE, this rule does not fire.
4. If there are no other rules in the list, the classifier uses the default label, which we defined as `reject_loan`.

Result: The classifier rejects the loan for this applicant.


## From scratch, in Python

Let's write a simple rule-based classifier in Python from scratch. First, we'll define the rules as a list of tuples. Each tuple contains a condition (as a function) and a label. We'll use plain Python functions for the conditions. 

```python
# A rule is a tuple: (condition_function, label)
# We'll use functions that take a feature dictionary and return True/False

def rule_1(features):
    return features['income'] >= 30 and features['debt'] <= 10

def rule_2(features):
    return features['income'] < 30 and features['income'] >= 20

rules = [
    (rule_1, 'approve'),
    (rule_2, 'review'),
]

def classify(features, rules, default_label='reject'):
    for condition, label in rules:
        if condition(features):
            return label
    return default_label

# Example applicant
applicant = {'income': 45, 'debt': 8}
print(classify(applicant, rules))  # Output: approve

# Another applicant
applicant2 = {'income': 25, 'debt': 5}
print(classify(applicant2, rules))  # Output: review
```


## The same thing, with a framework

```python
In practice, you might use scikit-learn's `DecisionTreeClassifier` or a rule-based library like `sklearn`'s `RuleBasedClassifier` (if it existed). For this beginner concept, the plain Python above is sufficient to show the mechanism. If you were to use a library, you might use `sklearn`'s `DecisionTreeClassifier` which internally creates rules, but for now, think of the library as a way to automatically learn rules from data, rather than hand-coding them. The underlying operation is the same: if-then conditions.
```


## What the code does

The code defines a list called `rules`. Each entry is a tuple: the first element is a function (like `rule_1`) that checks the conditions, and the second is the label to return if the conditions are met. The `classify` function takes a dictionary of features (e.g., `{'income': 45, 'debt': 8}`) and goes through each rule in order. It calls the condition function, and if it returns `True`, it returns the label. If none match, it returns the default label `'reject'`. This is exactly how a rule-based classifier works: sequential evaluation of rules. The example shows two applicants: one gets 'approve' because both conditions are true, another gets 'review' because income is between 20 and 30. This demonstrates how rules can be ordered—the first rule that matches wins.


## Common misconceptions

**Rules are always handwritten** — I have to write all the rules by hand, so this method is only for simple problems.

Rules can be automatically learned from data. Decision trees are a prime example: they are a set of rules induced from data, without human intervention. In fact: Rules can be either handcrafted or automatically induced. In machine learning, we often learn rules from data using algorithms like decision trees.

**Rules are as powerful as any model** — If I have enough rules, I can solve any classification problem perfectly.

Rules with simple thresholds on single features struggle with complex interactions between features. For example, a rule may not capture that the effect of income depends on debt level, unless you create many combinations, which becomes inefficient. In fact: Rules can approximate complex boundaries only with many rules, and they often fail on problems where numeric interactions matter, which is where models like trees or neural networks shine.

**'Bias' means unfair** — When someone talks about bias in a rule-based classifier, they mean the system is unfair.

In this context, 'bias' is a technical term for the threshold or default inclination, not social bias. For example, a rule might have a bias toward approving loans unless there is strong evidence against it. In fact: In rule-based classifiers, bias can refer to the default label or the threshold value, which is a design choice, unrelated to fairness (though unfairness can arise from biased rules).

**Rules must be in order** — The order of rules doesn't matter because they are independent.

Order matters because if two rules match, the first one wins. Changing the order changes the outcome. In fact: Rule order is critical; typically rules are ordered by priority, and the first matching rule determines the output.


## In short

A rule-based classifier is a set of explicit if-then rules that map input features to a label. It is simple, transparent, and can be handcrafted or learned from data. It works by checking conditions in order and returning the first matching label. While it is limited in capturing complex numeric interactions, it is widely used for its explainability and ease of deployment. Remember: rules are not always enough, but they are the perfect starting point to understand classification.


## Where this is used

Rule-based classifiers are used in many real-world systems, even though they may not be the most powerful models. They are especially valuable when transparency and explainability are critical.

- **Large language models (LLMs)**: LLMs like ChatGPT do not use rule-based classifiers internally, but they are used to generate and evaluate rules in agentic systems. For example, an LLM might be prompted to generate a set of rules for a specific task, and then a rule-based classifier executes those rules. This is a form of 'LLM as a rule generator'—the LLM writes the code, and the classifier runs it. This is common in automated decision-making systems where the LLM is asked to 'if this, then that' logic.
- **Small models on devices**: On resource-constrained devices (like a smart home hub), rule-based classifiers are ideal because they require no complex computations—just simple comparisons. For example, a smart thermostat might use rules like 'IF temperature > 30°C THEN turn on AC'. This is a tiny classifier that runs instantly without needing a neural network.
- **Retrieval-augmented generation (RAG)**: In RAG systems, a rule-based classifier might be used to decide whether to retrieve additional documents (e.g., 'IF the user asks for a specific fact, THEN retrieve from a knowledge base, ELSE generate from memory'). The rules act as a triage mechanism before the LLM is invoked.
- **Agentic systems that plan and call tools**: Agents often use rule-based logic to decide which tool to call next. For example, 'IF the user requests a weather report, THEN call the weather API'. The rules are simple and deterministic, which makes the agent's behavior predictable and auditable.
- **Everyday applied machine learning**: In business, banks use rule-based classifiers for initial screening of loan applications (as in our example). Fraud detection systems often have a set of rules (e.g., 'IF transaction amount > ₹1,00,000 AND merchant is abroad THEN flag for review') before a more complex model is applied. These rules are easy to update and explain to regulators.

What a developer writes: A developer might write a set of rules in a configuration file or directly in code, as we did above. In a library like scikit-learn, they might use `DecisionTreeClassifier` and then extract the rules using `tree.export_text()` to inspect them. But for pure rule-based systems, they might use a dedicated rule engine, like `rule-engine` in Python. The key point is that the logic is explicit and changeable without retraining.

What breaks if done badly: If the rules are too rigid, they fail on edge cases—a slight change in the input may lead to a completely wrong decision. If there are too many rules, the system becomes unmaintainable. And if the rules are induced from data without pruning, they can overfit and not generalise. A common failure is when two conditions conflict; the ordering of rules must be carefully chosen, or the system might make incorrect decisions.


---

Read this on the web: https://basavapurushottam.com/learn-ai/rule-based-classifiers
