# Empirical CDF

The step function that jumps by 1/n at each observation, estimating the cumulative distribution directly from data with no parametric assumption.

**Rests on:** Sampling Distributions

**Leads to:** Fitting Distributions

*Mathematical Foundations · Probability Theory*


## Why learn this

The empirical cumulative distribution function (ECDF) is the first step toward understanding how data becomes a model. It lets you answer questions like 'What fraction of customers spend less than ₹500?' directly from your data, without assuming any mathematical shape. This unlocks: quantiles (the values that split your data into percentiles), Q-Q plots (a visual check where you compare your data's spread against a known pattern to see if they match), and the bootstrap (resampling your existing data many times to estimate how much your results might vary) — all techniques used in everyday data analysis and in evaluating AI models. If you are ever asked 'How confident are you in that number?', the ECDF is where the answer starts.


## The intuition

Imagine you have collected the monthly spending (in rupees) of 10 customers: 200, 300, 300, 400, 500, 500, 500, 600, 700, 800. You want to know: 'What proportion of customers spend less than or equal to ₹450?' You could count: there are 4 customers who spend 400 or less (200, 300, 300, 400), so 4 out of 10, that is 0.4 or 40%. The ECDF is exactly this kind of counting, done for every possible threshold. For any amount X, the ECDF at X is the fraction of your data points that are ≤ X. Plot that fraction for every X, and you get a step-like curve: it stays flat between data points, jumps up by 1/n (here 1/10) at each observation, and ends at 1.0. It is a direct, honest picture of your data — no curve-fitting, no assumptions.


## An analogy

Think of the ECDF as a staircase built from your data. Each step is one observation (or a group of identical observations). The height of each step is 1/n, where n is the total number of data points — so with 10 data points, each step is 1/10 high. As you walk along the 'amount' axis from tiny amounts to huge amounts, the staircase climbs from near 0 to exactly 1. The height at any point X tells you the fraction of your data that is ≤ X. This staircase is the ECDF. Now, where does the analogy break? A regular staircase has even steps, but here steps are unevenly spaced along the axis — they occur exactly where your data points are. Also, if you have two identical values (like two customers spending 500), the step at that point is double height (2/n) because two data points share the same spot. The staircase is rough and jagged, not smooth. That is the point: it makes no assumptions about the underlying shape. It is the most honest possible description of the data you have.


## Definition

The empirical CDF is a step function that, at any value x, gives the proportion of data points less than or equal to x; it jumps by 1/n at each observed data point and gets closer and closer to the true underlying pattern as you collect more samples.


## How it connects

You are starting fresh — this is your first topic in probability and statistics. The ECDF is the foundation for understanding distributions without equations. Your library notes that it relates to 'Fitting Distributions' (where you assume a shape like a bell curve and fit parameters to it — the ECDF is what you compare against to check if that assumption is any good), 'Sampling Distributions' (how statistics bounce around across samples — the ECDF is a statistic itself), and 'Concentration Inequalities' (mathematical bounds that tell you how far the ECDF can stray from the true distribution — but that comes later). For now, think of the ECDF as the most direct way to summarise a dataset: it converts raw numbers into a curve that shows the full 'shape' of your data.


## The mathematics

The empirical cumulative distribution function (ECDF) is a way to describe the shape of a set of numbers you have collected. It answers the question: "What fraction of my data is less than or equal to a specific value?"

Imagine you have a list of n numbers. We call this total count n. For example, if you measured the heights of 10 people, n would be 10.

To build the ECDF, we follow these steps:

1. Sort the data from smallest to largest. This helps us see the order clearly.
2. Pick any number x. This x is just a test value you choose to check against your data.
3. Count how many numbers in your sorted list are less than or equal to x. Let's call this count C.
4. Divide that count C by the total number of items n. The result is the ECDF value at x.

We write this as:

F-hat-sub-n of x equals C divided by n.

Here, F-hat-sub-n is the name we give to this specific function for your dataset of size n. The hat symbol means it is calculated from observed data, not a theoretical model. The subscript n reminds us that the result depends on how many numbers you started with.

The value C can never be negative, and it can never be larger than n. Therefore, the fraction C/n will always be between 0 and 1. If no numbers are less than or equal to x, C is 0, and the ECDF is 0. If all numbers are less than or equal to x, C is n, and the ECDF is 1.

Graphically, the ECDF looks like a staircase. It starts at 0 on the far left. As you move right along the number line and pass each data point, the height of the graph jumps up. The size of the jump depends on how many data points are at that exact location. If all data points are different, every jump is exactly 1/n. If some points are repeated, the jump is larger.

Between the data points, the line is flat because no new data points have been passed yet. At each data point, the line goes up vertically. This vertical rise is what makes it a "step function."

The ECDF always ends at 1 on the far right, because eventually, every number in your dataset is less than or equal to any sufficiently large x.


## A worked example

Let us calculate the ECDF for a small set of data.

Data: Monthly spending (in rupees) for 10 customers:
200, 300, 300, 400, 500, 500, 500, 600, 700, 800.

Total number of data points, n = 10.

Step 1: Sort the data from smallest to largest.
The sorted list is:
200, 300, 300, 400, 500, 500, 500, 600, 700, 800.

Step 2: Calculate the ECDF at x = 450.
We look for all values in the sorted list that are less than or equal to 450.
The values are: 200, 300, 300, 400.
Count C = 4.
ECDF(450) = 4 divided by 10.
ECDF(450) = 0.4.

Step 3: Calculate the ECDF at x = 500.
We look for all values in the sorted list that are less than or equal to 500.
The values are: 200, 300, 300, 400, 500, 500, 500.
Count C = 7.
ECDF(500) = 7 divided by 10.
ECDF(500) = 0.7.

Step 4: Calculate the ECDF at x = 750.
We look for all values in the sorted list that are less than or equal to 750.
The values are: 200, 300, 300, 400, 500, 500, 500, 600, 700.
Count C = 9.
ECDF(750) = 9 divided by 10.
ECDF(750) = 0.9.

Step 5: Calculate the ECDF at x = 800.
We look for all values in the sorted list that are less than or equal to 800.
The values are: 200, 300, 300, 400, 500, 500, 500, 600, 700, 800.
Count C = 10.
ECDF(800) = 10 divided by 10.
ECDF(800) = 1.0.

Step 6: Describe the shape of the ECDF.
The graph starts at 0 for any x less than 200.
At x = 200, the count increases from 0 to 1. The jump size is 1/10 = 0.1.
The graph stays flat at 0.1 until x reaches 300.
At x = 300, there are two data points (the two 300s). The count increases from 1 to 3. The jump size is 2/10 = 0.2.
The graph stays flat at 0.3 until x reaches 400.
At x = 400, the count increases from 3 to 4. The jump size is 1/10 = 0.1.
The graph stays flat at 0.4 until x reaches 500.
At x = 500, there are three data points (the three 500s). The count increases from 4 to 7. The jump size is 3/10 = 0.3.
The graph stays flat at 0.7 until x reaches 600.
At x = 600, the count increases from 7 to 8. The jump size is 1/10 = 0.1.
The graph stays flat at 0.8 until x reaches 700.
At x = 700, the count increases from 8 to 9. The jump size is 1/10 = 0.1.
The graph stays flat at 0.9 until x reaches 800.
At x = 800, the count increases from 9 to 10. The jump size is 1/10 = 0.1.
For any x greater than 800, the ECDF remains at 1.0.


## From scratch, in Python

```python
# Pure Python implementation of the empirical CDF

data = [200, 300, 300, 400, 500, 500, 500, 600, 700, 800]

# Step 1: sort the data
data_sorted = sorted(data)

# Step 2: total number of points
n = len(data_sorted)

# Step 3: function to compute ECDF at a given x
def ecdf(x):
    # count how many points are <= x
    count = 0
    for value in data_sorted:
        if value <= x:
            count = count + 1
    return count / n

# Test on a few thresholds
print(ecdf(450))  # expected 0.4
print(ecdf(500))  # expected 0.7
print(ecdf(750))  # expected 0.9
```


## The same thing, with a framework

```python
import numpy as np

data = np.array([200, 300, 300, 400, 500, 500, 500, 600, 700, 800])

# NumPy's searchsorted returns the index of the first element greater than x,
# which equals the number of elements <= x.
def ecdf_numpy(x):
    return np.searchsorted(np.sort(data), x, side='right') / len(data)

print(ecdf_numpy(450))  # 0.4
print(ecdf_numpy(500))  # 0.7
print(ecdf_numpy(750))  # 0.9
```


## What the code does

This code introduces three Python building blocks: a list, a loop, and a function. A **list** is a container that keeps items in order — here, `data = [200, 300, 300, 400, 500, 500, 500, 600, 700, 800]` holds ten numbers. You can change a list later (add, remove, modify items), and lists are the default when you need to keep a collection of numbers. Next, `sorted(data)` returns a new list with the numbers in ascending order — essential for counting correctly. The `len(data)` function gives the length of a list (here, 10), which we store in `n`. Then we define a **function** `ecdf(x)` that takes a threshold `x` as input and returns the ECDF value. A function is a reusable block of code with a name, inputs (called parameters), and a returned result. Inside the function, we use a **for loop** — it repeats a block of code for each item in a list. Here, `for value in data_sorted:` goes through each number in the sorted list, and `if value <= x:` checks a condition (less than or equal to). If true, `count` increases by 1. After the loop, `return count / n` gives the fraction. Finally, we call our function with the `print` statement to see the results on screen.


## Common misconceptions

**The ECDF is a smooth curve.** — I expect the ECDF to be a smooth line like a bell curve, because that's what a distribution looks like.

The ECDF is built from actual data points. Between two data points, nothing changes, so the ECDF stays flat. It only jumps at the observed values. With many data points, the steps become tiny and it looks smoother, but it is always a step function. In fact: The ECDF is always a step function. It jumps by 1/n at each data point (or by k/n if k identical values exist). As n grows, the steps get smaller and the staircase approximates a smooth curve, but it never becomes perfectly smooth unless the underlying distribution is continuous and you have infinite data.

**The ECDF equals the true distribution.** — If I plot the ECDF, that IS the true distribution of the population.

The ECDF is an estimate based on the sample you happened to collect. A different random sample would give a different staircase. It is not the true distribution; it is a data-driven approximation. In fact: The ECDF converges to the true CDF as the sample size grows (a result called the Glivenko–Cantelli theorem). With small samples, the ECDF can differ substantially from the true distribution, especially in the tails. It is a powerful estimate, but always an estimate.

**The ECDF can predict values outside the observed range.** — If I have data between 200 and 800, I can use the ECDF to estimate the fraction of customers who spend more than 1000.

The ECDF is exactly 1.0 for any x beyond the maximum observation (and 0.0 for any x below the minimum). It knows nothing about what happens outside the observed range. In fact: The ECDF only describes the data you have. For x > max(data), ECDF(x) = 1, because all observed points are ≤ x. For x < min(data), ECDF(x) = 0. To go beyond the data, you need to fit a parametric distribution or use other extrapolation methods.

**The ECDF treats missing values as zero.** — If I have a missing data point, the ECDF just ignores it.

Ignoring missing values changes the denominator n. If you drop missing values, you are computing the ECDF on a different dataset, which can bias your estimate. In fact: Missing data should be handled explicitly: either drop them (and note that n is reduced) or impute them (and note that assumptions are made). The ECDF formula uses a specific n; changing which points are included changes the whole staircase.


## In short

The empirical CDF is a step function that estimates the true cumulative distribution directly from data — no assumptions, no curves. For any threshold x, it answers: 'What fraction of my data is ≤ x?' It converges to the true distribution as the sample grows, and it is the building block for quantiles, Q-Q plots, and the bootstrap. Remember: it is an estimate, not the truth; it cannot go outside your data range; and with small samples it is a rough staircase, not a smooth curve.


## Where this is used

The empirical CDF is a workhorse in applied machine learning, often hidden behind the scenes. Here is where it actually shows up:

- **Large Language Models (LLMs):** The ECDF is used to calibrate confidence scores. When an LLM outputs a probability (like 'I am 90% sure this answer is correct'), that number may be over- or under-confident. To calibrate, developers gather a set of predictions along with their predicted probabilities and true outcomes, then compute the ECDF of the predicted probabilities for correct vs. incorrect cases. This produces a calibration curve — if 90% of the cases where the model says '90% confident' are indeed correct, then the model is well-calibrated. The ECDF helps visualise and adjust that.

- **Small models on phones/laptops:** In tinyML and on-device models, memory and computation are limited. Instead of storing a full probability distribution (which may be a large table), developers can store only the ECDF of some reference data. When a new input arrives, the model computes a threshold and uses the ECDF to estimate whether the input is likely ('what fraction of training data had values this low or lower?'). This is used in anomaly detection — if the ECDF of a new value is extremely low (say, below 0.01), it is out of the ordinary.

- **Retrieval-Augmented Generation (RAG):** In RAG, a system retrieves relevant documents to answer a question. A common step is to check whether a retrieved document is 'relevant enough'. Developers build an ECDF from the similarity scores of known-good retrieved documents. If the new similarity score falls below a certain percentile (e.g., the 5th percentile of the ECDF), the document is likely irrelevant and gets filtered out. The ECDF provides a data-driven threshold instead of an arbitrarily chosen one.

- **Agentic systems (planning, tool calls):** Agents that decide whether to call a tool or take an action often have a confidence threshold. The ECDF of past decisions — e.g., the distribution of the 'risk score' when a tool call was helpful vs. harmful — helps set that threshold. If the agent's current risk score is in the bottom 10% of the ECDF of helpful calls, it is safe to proceed; otherwise, it might ask for more information.

- **Everyday applied ML (recommendation, forecasting, fraud detection):** 
  - **Fraud detection:** Model scores each transaction with a fraud probability. The ECDF of fraud scores helps decide a cutoff — if we want to catch 95% of frauds, we find the threshold corresponding to the 5th percentile of the ECDF of fraud scores, and flag anything above that.
  - **Recommendation:** When recommending products, the ECDF of predicted user ratings for, say, a movie can tell you what fraction of users would rate it above a certain level. This is used in 'percentile' recommendations.
  - **Forecasting:** Forecasters use the ECDF of forecast errors to estimate prediction intervals — the range that contains 80% of errors, read off from the ECDF's 10th and 90th percentiles.
  - **Anomaly detection in manufacturing:** ECDF of sensor readings flags rare values.

- **What a developer writes:** In practice, a developer rarely writes an ECDF from scratch; they use libraries. In Python, `numpy` has `np.searchsorted` which implements the counting operation. For plotting, `matplotlib` provides a direct `ecdf` function in `matplotlib.pyplot` (or they use `statsmodels`' `ECDF`. In R, `ecdf()` is built-in. In MATLAB (as in your library's reference), the `ecdf` function returns both the x (unique data values) and F (the cumulative fraction).

- **What breaks when done badly:** The most common mistake is treating the ECDF as if it were the true distribution. With small samples, the ECDF is a rough staircase that may be far from the true smooth curve — the jumps are coarse and the tails are poorly estimated. Claiming '40% of customers spend ≤ ₹450' based on 10 data points might be wildly off from the true proportion. Also, if you use the ECDF to set thresholds, you must account for its variability — that is where the bootstrap (a resampling technique that builds many ECDFs from your data) comes in. Ignoring that variability leads to overconfident decisions.

- **Where it does NOT appear:** The ECDF is not explicitly part of deep learning architectures (like the layers of a transformer). However, it appears in the evaluation and calibration of any model that outputs a confidence. So it is not inside the model, but it is in the evaluation loop.


---

Read this on the web: https://basavapurushottam.com/learn-ai/empirical-cdf
