# Context Precision and Recall

Retrieval metrics for RAG: precision measures how much retrieved context was relevant, recall whether the needed context was retrieved at all.

**Rests on:** Retrieval Quality

*LLMs and Generative AI · Retrieval-Augmented Generation*


## Why learn this

This concept gives you the two numbers that tell you whether your system for fetching documents is actually finding the right evidence for your AI assistant. Without these, you are flying blind — you cannot tell if a bad answer came from poor document retrieval or from flaws in the writing process itself. Mastering this lets you diagnose and fix retrieval problems, which is the first step toward building reliable AI systems that answer from your documents, not from guesswork. This unlocks further study in components like reordering fetched results, searching by both keywords and meaning, and using automated agents to find more data — all of which are about improving accuracy or completeness.


## The intuition

Imagine you ask an AI assistant a question, and it goes to a library to find the answer before responding. The library returns a stack of books, but you want to know: Did it give me the right books, and did it miss any important ones? Accuracy and completeness are exactly these two questions, measured with numbers. Accuracy: Of all the books it gave me, how many were actually relevant? Completeness: Of all the books that were relevant in the whole library, how many did it actually give me? High accuracy means no junk — every book it pulled is on-topic. High completeness means no misses — every book that had the answer was pulled. Most real systems trade one off against the other: if you pull only one book, accuracy might be high but completeness low; if you pull fifty, completeness goes up but accuracy drops. In AI document retrieval, we measure these on the snippets of text that get inserted into the model's context before it writes an answer.


## An analogy

Here is a richer analogy to carry through. Suppose you are a detective looking for clues about a crime. You have a file cabinet (the collection of all documents). You send your assistant to pull files. Accuracy: Of the files pulled, how many actually contain relevant clues? Completeness: Of all files in the cabinet that contain relevant clues, how many did the assistant bring back? If the assistant brings back 10 files but only 3 have relevant clues, accuracy is 30%. If there were 20 relevant files in the cabinet but only 3 were brought, completeness is 15%. Now here is the crucial asymmetry: if accuracy is low, you can throw away the irrelevant files — no harm to the investigation, just wasted space. But if completeness is low, you have missed clues entirely — you might never solve the case. That is why the library notes say 'Low completeness is unfixable downstream — the evidence simply is not there.' In AI, low accuracy wastes the model's available space (it gets distracted by irrelevant snippets), but low completeness means the model can only guess or invent facts. The analogy stops when you realise that in retrieval, you do not know the true set of relevant files — you can only estimate it by judging the retrieved set and maybe sampling the rest of the cabinet.


## Definition

Context accuracy is the fraction of retrieved text snippets that are relevant to the query; context completeness is the fraction of all relevant text snippets in the entire document collection that were successfully retrieved.


## How it connects

This concept belongs to retrieval evaluation, part of the broader field of locating information in large collections. It connects forward to evaluation frameworks like RAGAS, which break down quality into context accuracy, context completeness, answer relevancy, and answer correctness. It also underlies every technique you will learn later: reordering improves accuracy, searching by both keywords and meaning improves completeness by combining different search methods, automated agents improve completeness through iterative search, and splitting text directly affects both. Your library notes say that retrieval and answer quality are separately measurable and separately fixable — knowing these two numbers lets you fix the right half of a system that retrieves and then generates answers.


## The mathematics

The mathematics here is just fractions and sets — the branch is set theory and proportion, which is the simplest kind of statistics. You do not need algebra or calculus. The idea is: you have a group of things that you retrieved (the retrieved set). You also have a group of things that are actually relevant (the relevant set). These two groups overlap. Precision asks: of the things I retrieved, what fraction are in the relevant set? Recall asks: of the things that are relevant, what fraction did I retrieve? We calculate these by counting and dividing. You will first see these with a small example: pretend you have 5 retrieved items, and you mark each as relevant or not. Count the relevant ones, divide by total retrieved to get precision. For recall, you need to know the total relevant items in the entire corpus, which is often unknown in real life but easy in a toy example. Then you will see the symbols as shorthand for those counts.

Precision = (Number of retrieved items that are relevant) / (Total number of retrieved items)
Recall = (Number of retrieved items that are relevant) / (Total number of relevant items in the corpus)


## A worked example

```
Suppose we have a toy corpus of 6 chunks total. The query asks for a specific fact. We retrieve 4 chunks. Among those 4, we judge 3 as relevant. Also, in the entire corpus, there are 4 relevant chunks total (including 1 we did not retrieve).

Step 1: Retrieved items: [A, B, C, D]   # We have 4 chunks retrieved.
Step 2: Relevant among retrieved: A, B, C (3 out of 4)   # We judge each retrieved chunk against the query.
Step 3: Precision = 3 / 4 = 0.75   # 75% of what we retrieved was relevant.
Step 4: Total relevant in corpus: 4 (A, B, C, E)   # We know the entire corpus has 4 relevant chunks, but we only got 3 of them.
Step 5: Recall = 3 / 4 = 0.75   # We retrieved 75% of all relevant chunks.

Result: Precision is 0.75 (75%), recall is 0.75 (75%). In a real system, we would usually not know the total relevant count, so recall is estimated by judging a sample or using a test set.
```


## From scratch, in Python

```python
# Let's compute precision and recall for a simple retrieval example.

# We'll use Python lists to hold retrieved chunks and their relevance.
# A list is an ordered collection that can be changed.

retrieved = ["chunk_A", "chunk_B", "chunk_C", "chunk_D"]

# We manually mark which retrieved chunks are relevant (True) or not (False).
relevant_flags = [True, True, True, False]

# Count how many are relevant.
relevant_retrieved_count = 0
for flag in relevant_flags:
    if flag:
        relevant_retrieved_count = relevant_retrieved_count + 1

# Total retrieved is the length of the list.
retrieved_total = len(retrieved)

# We assume we know the total relevant in the corpus (from a test set).
relevant_total = 4  # known from our toy example

# Compute precision and recall.
precision = relevant_retrieved_count / retrieved_total
recall = relevant_retrieved_count / relevant_total

print("Precision:", precision)
print("Recall:", recall)

# Output:
# Precision: 0.75
# Recall: 0.75
```


## The same thing, with a framework

In a real RAG evaluation library like RAGAS, you would not compute precision and recall by hand. You would call a function that takes the retrieved chunks and the ground truth relevant chunks and returns scores. For example:

```python
from ragas.metrics import context_precision, context_recall

# Assume 'retrieved' is a list of document texts, 'ground_truth' is a list of relevant texts.
score_precision = context_precision.score(retrieved, ground_truth)
score_recall = context_recall.score(retrieved, ground_truth)
```

Underneath, these libraries use language models to judge relevance and then apply the same division we did above. The LLM judges whether a retrieved chunk is relevant to the query, producing the flags, and the metric computes the fraction.


## What the code does

This code introduces a few Python basics: a list (ordered collection), a for loop (repeating a block of code over items in a list), an if statement (checking a condition), and arithmetic division. We first create a list of chunk names, then a list of booleans (True/False) indicating relevance. The loop goes through each flag and counts the True values. Then we use `len()` to get the size of the list. Finally, we divide the counts to get precision and recall. In a real retrieval system, you would get the chunks from a search API and the relevance flags would come from human judgement or a benchmark.


## Common misconceptions

**Precision and recall are always inversely related** — If I want to increase precision, recall must drop, and vice versa.

In some systems, retrieving more can increase both, especially if the initial retrieval misses many relevant items. The tradeoff is a tendency, not a law. With better retrieval algorithms, you can improve both simultaneously. In fact: Precision and recall can both improve with a better retrieval method. The tradeoff appears when you only change the number of retrieved chunks without improving the ranking.

**High recall is always better than high precision** — The most important thing is to retrieve everything relevant, even if there's junk.

Low precision wastes the model's context budget — irrelevant chunks distract it and can degrade answers. The two metrics must be balanced based on the use case. In fact: Both matter. The ideal is high precision and high recall. The relative importance depends on the cost of missing evidence versus the cost of distraction.

**Recall is easy to measure accurately** — I can just count the relevant retrieved chunks and divide by the total relevant in the corpus, and that's my recall.

In real corpora, you do not know the total number of relevant documents — it is often infinite or at least extremely large. You can only estimate recall using a test set or by exhaustive labelling of a small corpus. In fact: Recall is usually an estimate. You need a ground-truth set of relevant documents for a set of queries, often built by human labelling, to calculate it.

**Precision and recall are only for documents** — These metrics only apply to whole documents.

In RAG, we measure precision and recall at the chunk level or even at the span level (specific passages), which is more granular. In fact: You can compute these metrics for any retrieval unit — documents, chunks, sentences, or even token spans. In RAG, chunk-level is standard.

**Context precision and recall are the same as answer precision and recall** — If the final answer is good, retrieval must have been good.

The generator might produce a correct answer from parametric memory even if retrieval was irrelevant, or it might fail to use good evidence. In fact: Context precision and recall measure retrieval quality independently of answer quality. You need separate metrics for answer relevancy and correctness.


## In short

Context precision tells you what fraction of what you retrieved was relevant; context recall tells you what fraction of all relevant content you managed to retrieve. Low recall means the evidence is missing—unfixable downstream. Low precision means the model's context is cluttered—distracting and wasteful. Measure both on a test set before trusting your RAG system.


## Where this is used

These two metrics are used everywhere in production RAG systems. In large language models, when people build a chat assistant over their company's documents, they measure context precision and recall on a test set before deployment. If recall is low, they know the model is missing evidence, so answers may hallucinate. They might then switch from vector search to hybrid search (adding keyword matching) to improve recall. If precision is low, they might add a reranking step after retrieval to move relevant chunks to the top and drop irrelevant ones. In small models running on devices, such as a phone assistant that retrieves from a local database, precision and recall are still measured, but the chunk size might be smaller to fit memory, which can hurt recall; engineers tweak the chunk size to balance. In agentic systems that plan and call tools, the model might retrieve a few chunks, see that recall is poor (by evaluating the retrieved content against the query), and issue another search — that is agentic retrieval. In everyday applied work like recommendation systems, precision and recall are used to measure how many recommended items were relevant (precision) and how many relevant items were recommended (recall). In fraud detection, recall matters most because missing a fraudulent transaction is costlier than a false alert. For speech and vision, these metrics appear in retrieval-based systems that fetch images or audio clips relevant to a query. Developers write code to compute these metrics on a validation set before each model update, and they use tools like RAGAS or custom scripts. When done badly, engineers might only measure precision (easy to do) and miss that recall is terrible, causing the system to silently fail on many queries. The library notes emphasise that low recall is unfixable downstream because the evidence is simply not in the context; no amount of prompt engineering can generate missing information.


---

Read this on the web: https://basavapurushottam.com/learn-ai/context-precision-and-recall
