# Contract Testing

Verifying that a provider and consumer agree on their interface, without running both together end to end.

**Rests on:** API Testing

*Software Engineering · Software Testing*


## Why learn this

Contract testing lets you update one part of a software system without breaking the parts that talk to it. Imagine a company split into two teams: Team A builds the login screen, and Team B handles the user database. They agree on a rule: when you send a username and password, the database must reply with a success token or an error code. Contract testing verifies that both sides stick to this rule every time they change their code. Without it, Team B might update their code in a way that confuses Team A's screen, forcing both teams to pause work while they fix the mismatch. This allows independent deployment: Team B can release updates quickly without waiting for Team A to check their changes, keeping the whole system moving.


## The intuition

Imagine you are in a restaurant. The kitchen (provider) and the waiter (consumer) both need to agree on what the menu says — the names of dishes, the prices, what comes with each. They do not need to cook together in the same kitchen to check they agree. The menu itself is the contract. If the kitchen changes the recipe without updating the menu, the waiter will bring out the wrong dish — that is a broken contract. Contract testing is the practice of checking that both sides stick to the menu, but each side checks separately, without the other being present. The kitchen tests that its dishes match the menu. The waiter tests that the menu tells them everything they need to write the order. If both pass, they can be deployed independently — changes on one side do not break the other as long as the contract holds.


## An analogy

Think of two colleagues working on a joint report. One writes the introduction, the other the conclusion. They agree that the introduction will end with a certain paragraph that the conclusion will refer to. Instead of meeting every day and reading each other's drafts, they write down a shared outline: 'The introduction will state the main finding in its final paragraph.' Each writes their part against that outline. If one changes the numbering of the sections, the other's references break. Contract testing is this outline — written down, checked by both sides independently. The key point is that neither reads the other's full work. They only check that their own part matches the outline. The analogy stops working when the contract is not exhaustive — in real life, two people can misunderstand the outline, whereas a contract in testing is precise and machine-checkable. Also, a contract does not capture the full behaviour, only the interface — like the outline captures structure, not the style.


## Definition

Contract testing verifies that a service consumer and a service provider agree on the format and semantics of their interactions, by each testing against a shared contract, independently, without running the other side.


## How it connects

This concept builds directly on API Testing — your library notes say contract verification catches integration breakage early. It sits beside Exploratory Testing, which finds what scripts cannot, and contrasts with end-to-end testing, which is the slow top of the pyramid. The testing pyramid in your notes says many fast unit tests, fewer slow end-to-end ones — contract testing sits between the two, fast but more integrated than a unit, yet not requiring the whole system. It also supports Shift-Left Testing, moving defect detection earlier, because it runs in every build. It is the practice that makes the pyramid practical at the service level.


## The mathematics

The mathematics here is simple: it is about checking equality and set membership. You have two sets of requirements — what the consumer expects and what the provider offers. The contract is a set of rules that both must satisfy. The checking is done by comparing each rule against each side's actual behaviour, which is a yes/no operation — does the actual response match the expected shape and values? This is not about calculating a number; it is about boolean logic — true or false. You are verifying that a structure matches a template, which involves checking that every required field is present, that the types match, and that the values fall within allowed ranges. This is arithmetic of comparison — does this equal that, is this greater than that, is this in this set? That is the whole mathematical operation.

For each field f in contract: σ_provider(f) = true if provider's response matches the contract's type and constraints, else false. Contract satisfied if and only if ∀f ∈ contract, σ_provider(f) = true.


## A worked example

```
We have a contract that says a 'GET /user' endpoint returns a JSON object with fields: 'id' (integer), 'name' (string), and 'age' (integer, optional). The consumer expects these fields.

Step 1: Check: actual response has 'id' field → true   # We look for the presence of the field.
Step 2: Check: 'id' is integer → true (actual: 123)   # We check the type.
Step 3: Check: 'name' is string → true (actual: 'Alice')   # We check the type.
Step 4: Check: 'age' is optional — if present, must be integer → present as 30, integer → true   # Optional fields are only checked if present.

Result: All checks pass, so the provider satisfies the contract. If any check returned false, the contract is broken.
```


## From scratch, in Python

```python
# We will write a simple contract checker in plain Python.
# First, let's learn the building blocks: a dictionary, a list, and a function.

# A dictionary maps keys to values, like a phone book.
user_response = {"id": 123, "name": "Alice", "age": 30}

# A list is an ordered collection, like a shopping list.
required_fields = ["id", "name"]

# A function is a reusable block of code that takes inputs and returns an output.
def check_contract(response, required, optional):
    # Loop over each field in the required list
    for field in required:
        # If the field is missing from the response dictionary, the contract fails
        if field not in response:
            return False
    # Loop over each optional field
    for field in optional:
        # If the field is present, we must check its type matches
        if field in response:
            # The optional 'age' must be an integer
            if field == "age" and not isinstance(response[field], int):
                return False
    return True

# Test it
result = check_contract(user_response, required_fields, ["age"])
print("Contract satisfied:", result)
```


## The same thing, with a framework

```python
In practice, you would use a library like Pact or Spring Cloud Contract instead of writing your own. For example, with Pact, the contract is defined as a Python dictionary and verified using the `pact-python` library, but underneath it does exactly the same checks: it compares the actual provider response against the expected fields and types.
```


## What the code does

Line by line: First, we create a dictionary `user_response` that holds the provider's actual return value. Then `required_fields` is a list of field names that must exist in the response. The function `check_contract` takes these and also a list of optional fields. Inside, a `for` loop goes through each required field and checks if it is a key in the dictionary using `not in`. If any missing, it returns False. Then it checks optional fields — if present, it verifies the type with `isinstance`. Finally, we call the function and print the result. This is the core of contract testing: comparing expected versus actual.


## Common misconceptions

**Contract testing is the same as end-to-end testing** — If my end-to-end tests pass, I don't need contract tests.

End-to-end tests run both services together, so they don't tell you which side broke when something fails, and they are slow. Contract tests run each side in isolation, so failures point directly to the responsible side. In fact: Contract testing is complementary to end-to-end: it verifies each side independently against a shared agreement, which is faster and pinpoints the breakage.

**A contract is just an API specification** — Swagger documentation is the same as a contract.

Documentation is static and often out of date. A contract is executable — it is used to automatically verify both sides in CI. In fact: A contract is a machine-readable and verifiable agreement; documentation just describes it.

**If the consumer changes, the provider must change too** — Every change on one side requires updating the other.

If the change is backward-compatible (e.g., adding a new optional field), the contract still holds. Contract testing ensures old consumers continue to work. In fact: Contract testing allows independent evolution as long as the existing contract is honoured.

**Contract testing replaces all integration testing** — If all contracts pass, the system will work end to end.

Contracts verify interface agreement, not behaviour. They do not catch bugs in business logic that emerge only when services interact. In fact: Contract testing is part of a testing strategy, alongside unit and end-to-end tests.


## In short

Contract testing verifies that a provider and consumer agree on their interface without running both together. Each side tests against a shared contract, which makes independent deployment safe. It is fast, catches integration breakage early, and complements end-to-end testing.


## Where this is used

In large language model APIs, a provider like OpenAI returns a structured JSON with certain fields (e.g., 'choices', 'usage'). Contract testing ensures that the client code (e.g., a Python script calling the API) correctly handles changes in that structure without breaking. For small models on devices, when a mobile app loads a model configuration, contract testing verifies the model schema is consistent. In retrieval-augmented generation, the embedding service and the database must agree on vector dimension — contract testing catches if one returns 384-dim and the other expects 768-dim. In agentic systems that plan and call tools, each tool has an input/output contract; contract testing verifies the agent's calls are valid. In everyday ML, a prediction service's response (e.g., 'score' and 'label') must match what the frontend expects. A developer writes contract tests in Pact or by asserting against a schema, and if done badly, integration breaks only in production when both sides deploy.


---

Read this on the web: https://basavapurushottam.com/learn-ai/contract-testing
