# Promptfoo and LightEval

Open-source harnesses for running structured evaluations over prompts and models with reproducible configuration.

**Rests on:** Evaluation Datasets

*LLMs and Generative AI · LLM Evaluation*


## Why learn this

You are learning this because it is the first tool you will use to actually test an AI system you have built. Without it, you can only guess whether your changes helped or broke things. By the end of this page you will be able to: write a small evaluation file that runs a list of test prompts through a model and checks the answers; run that file from your terminal and see a pass/fail report; and set it up so that every time you change your code, the evaluation runs automatically. This is the same skill that professional teams use to stop a new model update from silently making their chatbot worse.


## The intuition

When you build an AI system, you want to know if it is getting better. You cannot just ask 'is it good?' because 'good' is too vague. Instead you write down a list of questions and the answers you expect, then you run your system on that list every time you make a change. If a change makes it answer a question wrong that it used to get right, you know instantly you broke something. This is what an evaluation harness does for you. Promptfoo and LightEval are two open-source tools that do this. They are not the only ones, but they are popular and they show the core idea: you write a config file that says what model to call, what prompts to send, and how to check the responses. The tool runs all of that for you and prints a report. The magic is that the config file is text, so it can be saved, shared, and run again later. That is what makes your testing reproducible: anyone can run the same file and get the same test.


## An analogy

Imagine you are a chef creating a new recipe. You have a taste-tester (your AI model). You do not just ask the tester 'is it good?' because that is subjective and varies day to day. Instead you write a fixed tasting card: a list of dishes with the expected notes (sweet, salty, spicy). Each time you change the recipe, you serve the same tasting card and mark pass or fail for each dish. This is exactly what an evaluation harness does. The config file is the tasting card. The model is the taste-tester. The checks are the marks. Now, the analogy breaks down: a recipe change might improve some dishes and ruin others, and the tasting card captures that; but here you have hundreds of prompts, not ten dishes, and the checks are often automated, not human. Also, the model's answers are not just taste notes but free text, so the checks must be smart enough to tell if 'the sauce is too salty' means 'fail' or 'pass'.


## Definition

Promptfoo and LightEval are open-source testing tools that let you define a list of prompts and expected outcomes in a configuration file, run them against a model, and automatically decide whether each response passes or fails, so you can catch regressions and compare versions.


## How it connects

You have not studied any other topics yet. This is your entry point into AI evaluation. After this, you will learn about Evaluation Datasets (the prompts and expectations themselves), Deterministic Validators (the simple checks that give pass/fail), and then LLM-as-Judge (when you use a model to grade another model). Promptfoo and LightEval are the harnesses that bring those ideas together into something you run.


## The mathematics

When we evaluate a Large Language Model, we are essentially measuring how often it gets things right. The mathematics involved is basic arithmetic: counting items and dividing one number by another. We do not need algebra, calculus, or complex formulas. We only need to know how to add whole numbers and divide.

Imagine you have a list of questions (prompts) that you want the model to answer. Let us say there are 20 questions in total. You send each question to the model and get an answer back. For each answer, you perform a simple check: does the answer meet your requirement? If it does, we call this a "pass." If it does not, we call it a "fail."

To find out how well the model performed overall, we follow three steps.

Step 1: Count the total number of questions you asked. Let us call this number T. In our example, T equals 20.

Step 2: Count how many of those answers were passes. Let us call this number P. Suppose that out of the 20 answers, 15 met the requirement and 5 did not. So, P equals 15.

Step 3: Divide the number of passes by the total number of questions. This gives us the pass rate. We write this as a fraction: P divided by T.

In our example, we divide 15 by 20.

15 divided by 20 equals 0.75.

To express this as a percentage, we multiply by 100. 

0.75 times 100 equals 75.

So the pass rate is 75%. This means the model answered correctly 75% of the time.

This calculation works for any number of prompts. If you have T prompts, and P of them pass, the formula is:

Pass Rate = P divided by T.

Sometimes, instead of just counting passes, we might assign a score to each answer. For example, if an answer is partially correct, it might get 0.5 points. If it is fully correct, it gets 1 point. If it is wrong, it gets 0 points. To find the average score, we add up all the individual scores and divide by the total number of prompts T.

Sum of Scores = Score for prompt 1 plus Score for prompt 2 plus ... plus Score for prompt T.

Average Score = Sum of Scores divided by T.

The pass rate is just a special case where every score is either 0 or 1. The average score then tells us the fraction of prompts that passed.


## A worked example

Let us work through a specific example with real numbers to see how the calculation works step by step.

Suppose you are testing a model on a small set of 3 prompts. You want to check if the model's response contains the word "refund". This is your rule for a pass.

Prompt 1:
Question: "What is your refund policy?"
Model Response: "You can get a refund within 30 days."
Check: Does the response contain the word "refund"? Yes, it does.
Score for Prompt 1 = 1 (because it passed).

Prompt 2:
Question: "How do I cancel my account?"
Model Response: "Call support at 555-0199."
Check: Does the response contain the word "refund"? No, it does not.
Score for Prompt 2 = 0 (because it failed).

Prompt 3:
Question: "Return policy?"
Model Response: "Items can be returned within 30 days for a full refund."
Check: Does the response contain the word "refund"? Yes, it does.
Score for Prompt 3 = 1 (because it passed).

Now we calculate the total pass rate.

Step 1: Find the total number of prompts, T.
T = 3.

Step 2: Find the total number of passes, P.
P = Score for Prompt 1 + Score for Prompt 2 + Score for Prompt 3.
P = 1 + 0 + 1.
P = 2.

Step 3: Calculate the pass rate by dividing P by T.
Pass Rate = 2 divided by 3.

To do this division, we can write it as a fraction: 2/3.
When you divide 2 by 3, you get 0.6666... (the digit 6 repeats forever).

Step 4: Convert to a percentage.
Multiply 0.6666... by 100.
0.6666... times 100 equals 66.666...%

We usually round this to one decimal place for reporting.
66.666...% rounds to 66.7%.

Result: The evaluation reports a pass rate of 66.7%. This tells you that the model met your expectation on 2 out of the 3 prompts.


## From scratch, in Python

```python
import json

def evaluate(prompts, check_func):
    results = []
    for prompt in prompts:
        # In a real system, you would call your model here.
        response = "This is a mock response to " + prompt
        passed = check_func(response)
        results.append({"prompt": prompt, "response": response, "passed": passed})
    return results

def contains_refund(response):
    return "refund" in response

prompts = [
    "What is your refund policy?",
    "How do I cancel?",
    "Return policy?"
]

results = evaluate(prompts, contains_refund)
for r in results:
    print(r["prompt"], "->", "PASS" if r["passed"] else "FAIL")

passed_count = sum(1 for r in results if r["passed"])
pass_rate = passed_count / len(prompts)
print(f"Pass rate: {pass_rate:.2f}")
```


## The same thing, with a framework

In Promptfoo, you write a YAML file:

```yaml
prompts:
  - "What is your refund policy?"
  - "How do I cancel?"
  - "Return policy?"

tests:
  - vars:
      prompt: "What is your refund policy?"
    assert:
      - type: contains
        value: "refund"
  - vars:
      prompt: "How do I cancel?"
    assert:
      - type: contains
        value: "refund"
  - vars:
      prompt: "Return policy?"
    assert:
      - type: contains
        value: "refund"
```

Then run `promptfoo eval`. Underneath, it reads the config, runs the model for each prompt, applies the checks, and prints a pass/fail table just like the Python above.


## What the code does

The core is a loop: for each prompt, call the model (here mocked), apply a check function, and store the result. The check function is a simple `in` operator that checks if a substring is present. Then we count passing results and divide by total to get the pass rate. In the Promptfoo config, we declare the same logic declaratively: each test lists the prompt and an assertion. The tool runs the loop for you.


## Common misconceptions

**Evaluation is unnecessary for small changes** — I just changed one line in the prompt, it can't break anything. I don't need to run an evaluation every time.

Even a tiny change like adding a comma can alter the model's output and cause a regression. Without a test, you will not notice until users complain. Evaluation is cheap; a regression in production is expensive. In fact: Run the evaluation on every change, no matter how small, especially in a CI pipeline. That is the whole point of a regression gate.

**A pass rate of 100% means it is perfect** — If my evaluation passes all tests, my system is bug-free and works for everyone.

Your test set is only a sample. It may not cover edge cases, unusual wording, or adversarial inputs. A 100% on a small, biased set is misleading. In fact: Your evaluation is as good as your dataset. Keep expanding the dataset with real user queries and edge cases, and treat 100% as a sign to add more tests, not to declare victory.

**The tool is the hard part** — I just need to learn how to configure promptfoo and I will have good evaluations.

The tool just runs your checks. If your dataset is bad (few prompts, unrealistic ones), the evaluation is worthless regardless of the tool. In fact: Focus most effort on building a high-quality dataset that reflects real user behavior and covers important failures. The tool is just an execution engine.

**One evaluation run is enough** — I ran the evaluation once, it passed, so my system is done.

Models can be nondeterministic (they sample). Also, your requirements will change. One run does not guarantee robustness under different seeds or future changes. In fact: Run evaluations regularly (in CI) and on the same dataset to detect drift and variance. Also, run multiple times with different random seeds to see if results are stable.


## In short

Promptfoo and LightEval are tools that run a fixed set of prompts through a model and check each response against rules you define. The output is a pass rate (passes / total). The key is the dataset: a good evaluation set is worth more than any tool. Always integrate evaluation into your workflow so you catch regressions before your users do.


## Where this is used

You will use evaluation harnesses like this in real AI work in several ways.

**In large language models (LLMs):** When a team is developing a customer-support chatbot, they write a set of 50–100 typical user questions (like 'What is your refund policy?') along with checks (like 'response must mention refund and not mention illegal activity'). Every time they update the model or the prompt template, they run the suite. If the pass rate drops, they know the change broke something. The mechanism is that the config file pins the test set, and the harness runs it against any model version, making regressions visible immediately.

**In small models on devices (TinyML):** For a keyword-spotter that runs on a phone, the dataset is audio clips, not text. But the idea is the same: a fixed set of clips, a check that the correct keyword was detected, and a pass rate. LightEval is more geared toward large models, but you can adapt the concept to any model that outputs a label. The developer writes a simple script that loads the model, runs the audio through it, and compares the predicted label to the true label.

**In retrieval-augmented generation (RAG):** RAG systems retrieve documents and then generate an answer. You want to check that the retrieval returns relevant documents and that the answer uses them. An evaluation prompt might be 'What is the return policy?' and a check might be 'response must contain the phrase '30 days' from the retrieved document'. The harness runs the whole pipeline (retrieval + generation) and checks the final output. This catches failures where the retrieval returns irrelevant docs or the model ignores them.

**In agentic systems that plan and call tools:** Agents decide which tool to call, call it, and use the result. You can write checks like 'the agent called the calculator tool with two numbers' or 'the final answer is a number between 0 and 100'. The harness runs the agent loop for each test prompt and verifies the tool call sequence and final answer. This is more complex because the agent may take multiple steps, but the harness just runs the whole thing and checks the end state.

**In everyday applied ML (recommendation, vision, forecasting, fraud):** For a recommendation system, you might have a list of user IDs and expected item categories. You run the model and check if the recommended item matches the expected category. For fraud detection, you have transactions with labels (fraud/not fraud). The harness runs the model on each transaction and checks the predicted label against the true one. For forecasting, you have time series and you check if the predicted value is within a tolerance of the actual value.

**What a developer actually writes:** In practice, a developer writes a configuration file (like YAML or Python) that declares the prompts, the model to call, and the assertions. They also write the model invocation code (a function that calls an API) and the assertion functions. They run the harness in their CI pipeline so that every pull request is tested. If a change drops the pass rate below a threshold, the build fails and the developer fixes it before merging.


---

Read this on the web: https://basavapurushottam.com/learn-ai/promptfoo-and-lighteval
