# Spec Drift

Implementation gradually diverging from the specification, so the document stops describing the system.

**Rests on:** Spec-to-Code Pipeline

*Software Engineering · Spec-Driven Development*


## Why learn this

Spec drift is the silent killer of AI projects. When the written specification and the actual code quietly drift apart, every downstream decision — testing, debugging, adding features, handing off to another team — becomes guesswork. By mastering this concept, you will be able to detect drift early, understand why it happens, and know how to keep your documentation honest. This unlocks the next stage of your learning: Requirements Traceability, where you will learn to build the links that make drift impossible to hide. From there, you will move on to Acceptance Criteria and Review Gates, turning 'is the code right?' from a feeling into a testable, checkable process.


## The intuition

Imagine you are following a recipe to bake a cake. The recipe says 'add 200g of flour, then 2 eggs, then bake for 30 minutes.' But halfway through, you decide to add an extra egg because the mixture looks dry. Then you bake it for 40 minutes because the oven seems slow. The cake turns out fine — but if you bake it again next week, you will follow the recipe exactly as written, not as you actually made it. The second cake might be dry or too eggy. The recipe no longer describes what you actually do. That is spec drift: the written document (the recipe) and the actual implementation (your baking) slowly diverge. Each small deviation feels harmless — an extra egg, a few extra minutes — but together they mean the document has stopped telling the truth. And the danger is that everyone trusts the document. If someone new reads the recipe, they will follow it exactly, and get a different result than you did. In software, spec drift is the same: a requirement states 'the search should return results in under 2 seconds.' The developer optimises one query, but then adds a feature that makes it 3 seconds. Then another developer, seeing the spec, assumes 2 seconds is still the target and spends days debugging a problem that no longer exists. The document is trusted, but it is wrong. Drift is invisible because nothing crashes — the system still works, just differently than specified. The tragedy is that by the time you notice, the gap is so large that fixing it means rewriting either the code or the spec, and both are expensive. In AI systems, where models are retrained, prompts are tweaked, and data pipelines change constantly, drift happens even faster. A prompt that says 'classify this email as spam or not spam' might drift as the model is fine-tuned, or as the data distribution shifts. The spec says one thing; the system does another. The longer you ignore the gap, the harder it is to close. The fix is not to avoid all change — change is inevitable and often good — but to make drift detectable. You need a way to compare the spec against the implementation regularly, like checking the recipe against the cake you actually made.


## An analogy

Think of a city's official map and its actual streets. The map is the specification — it describes the roads, their names, their one-way directions. When the city builds a new road, or changes a street to one-way, someone updates the map. But if the map is not updated, a driver following it will hit a dead end, or drive the wrong way down a one-way street. Now, the map is not just unhelpful — it is dangerous, because the driver trusts it. Spec drift is the gap between the map and the streets. At first, the difference is tiny: a new roundabout, a renamed street. Nobody notices because the old road still exists, just slightly differently. But then the city builds a bypass, and the old road becomes a cul-de-sac. The map still shows it as a through-road, and a driver following the map ends up stuck. The cost of a wrong map is not just one wrong turn — it is every driver who trusts it, and every journey that is planned around it. The same happens in software. A specification says 'the login endpoint accepts email and password.' The developer changes it to accept a token instead, but does not update the spec. Now every test, every review, every new developer reading the spec is planning around the old design. The system works, but the map is wrong. Where does the analogy break down? In a city, the map is a physical object that exists separately from the streets. If it is wrong, you can throw it away and draw a new one. In software, the specification and the code are intertwined — the code is the implementation of the spec, and the spec is the explanation of the code. Correcting a wrong map is easy; correcting a wrong spec requires deciding which one is the source of truth, and that is a hard, political decision. Also, in a city, streets change slowly. In software, the code changes every day, every commit. Drift is not a rare event — it is the default. The map is always slightly out of date. That is why you need a process, not just goodwill, to keep them aligned.


## Definition

Spec drift is the gradual, often unnoticed divergence between a written specification and the actual implemented system, so that the document stops accurately describing what the code does.


## How it connects

You have not studied anything yet, so this is your starting point. But know that spec drift is the reason the entire practice of Spec-Driven Development exists. The spec-to-code pipeline, acceptance criteria, and review gates are all defensive measures against drift. When you get to Requirements Traceability, you will learn the tool that makes drift visible: a traceability matrix that links each requirement to the tasks, tests, and code that implement it. That is the practical answer to the problem this concept describes. For now, understand the disease before you learn the cure.


## The mathematics

This concept is not mathematical — it is about workflow and consistency. But there is one quantitative way to think about drift: a measure of how far the specification and the implementation have diverged. Imagine you have a list of ten requirements in the spec. You look at the code and ask of each one: is this fully implemented? Maybe six are fully there, three are partial, and one is missing. That is a gap. The branch of mathematics that helps here is arithmetic — just counting and comparing. No algebra, no calculus. The key idea is a ratio: how many requirements does the code actually satisfy, out of the total? If you have ten requirements and the code satisfies six, that is six out of ten, or 60%. As time goes on, the spec says ten requirements, but the code might only satisfy five, or the code might do things the spec never mentioned. The gap is the difference between what the spec promises and what the code delivers. It is a simple subtraction, but it is powerful because it forces you to count. Drift becomes visible when you actually count, requirement by requirement, what is true. Without counting, it is just a vague feeling that 'things are not quite aligned.' With counting, you can say: 'We are at 60% alignment, and it is getting worse by 2% per week.' That number is the early warning you need.

Let T be the total number of requirements in the specification, and I be the number of those requirements that are fully implemented and working in the code. Then:
   alignment = I / T × 100%
   spec drift = 100% − alignment
Where T is 'total requirements' (a count), I is 'implemented requirements' (a count), and the result is a percentage. This formula is simple arithmetic — you only need to count and divide — but it gives you a number you can track over time. If drift increases each week, you know you are falling behind.


## A worked example

```
Suppose your specification lists 5 requirements for a customer support chatbot:
  R1: Respond to greetings (e.g., 'Hi')
  R2: Answer FAQs about shipping
  R3: Escalate to a human when the user asks for it
  R4: Remember the user's name within a session
  R5: Apologise when it cannot answer

Step 1: Total requirements in spec = 5   # Start with the count from the written document.
Step 2: Requirements actually implemented in code = 3 (R1, R2, R3)   # R4 and R5 were never built; the spec still lists them.
Step 3: Requirements implemented but not in spec = 1 (R6: 'Tell a joke')   # The developer added a joke feature that was never specified.
Step 4: Alignment score = 3 ÷ 5 = 0.6 = 60%   # Only 60% of the specified requirements are actually in the code.
Step 5: Spec drift = 100% − 60% = 40%   # The specification is 40% out of date — it promises five things, but delivers three, and does one extra thing no one asked for.

Result: The spec and the code are 40% apart. If you were to hand this spec to a new developer, they would try to build R4 and R5, and would be surprised by the joke feature. Drift is measurable, and this number is how you catch it early.
```


## From scratch, in Python

```python
# We will write a small script that compares a specification to code.
# First, we need a way to store the specification. A list is perfect:
# it keeps items in order and can be changed later.

spec_requirements = ["Respond to greetings", "Answer FAQs", "Escalate to human", "Remember name", "Apologise"]

# Now we need a list of what the code actually does. We will manually
# fill this in, but in a real project it might come from reading the code.

implemented_features = ["Respond to greetings", "Answer FAQs", "Escalate to human", "Tell a joke"]

# To measure drift, we count how many spec requirements are in the
# implemented list. We need a loop to go through each requirement and
# check if it is there.

count_implemented = 0          # start at zero
missing = []                   # an empty list to collect what is missing

for req in spec_requirements:  # 'for each requirement in the spec list'
    if req in implemented_features:   # 'if it is in the implemented list'
        count_implemented = count_implemented + 1
    else:
        missing.append(req)    # add to the missing list

# Now count the extras: things in the code that are not in the spec.
extras = []
for feature in implemented_features:
    if feature not in spec_requirements:
        extras.append(feature)

# Now compute the alignment percentage.
total = len(spec_requirements)   # len() gives the number of items in a list
alignment = (count_implemented / total) * 100
drift = 100 - alignment

print("Spec requirements:", spec_requirements)
print("Implemented features:", implemented_features)
print("Missing from code:", missing)
print("Extra features not in spec:", extras)
print(f"Alignment: {alignment:.1f}%")
print(f"Spec drift: {drift:.1f}%")
```


## What the code does

Line by line: we start with a Python list holding the requirements. A list is just an ordered collection — write the items in square brackets, separated by commas. We create another list for what the code actually does. This second list is filled by hand here, but in a real project you might generate it by scanning the codebase for function definitions or test names. The next block uses a for loop. A for loop repeats an action for each item in a list. The line `for req in spec_requirements` means 'take the first item, run the indented code, then take the second item, run it again, and so on.' Inside the loop, we use `if req in implemented_features` to check whether that requirement appears in the other list. The `in` operator checks membership — it returns True or False. If True, we add 1 to the counter `count_implemented`. If False, we add it to the `missing` list using `.append()`. After that loop, we do a second loop to find extra features in the code that are not in the spec — things someone added without telling the spec. Then we compute the percentage. `len(spec_requirements)` gives the number of items in the list — here it is 5. We divide the count by that total, multiply by 100 to get a percentage, and subtract from 100 to get drift. Finally, we print everything. The `f"..."` string lets us insert the number with one decimal place. When you run this, you see the missing items, the extras, and the drift percentage. That is the whole point: making the invisible visible.


## Common misconceptions

**Drift only happens if developers are sloppy** — I am a careful developer, so my spec and code will never drift.

Drift is not a moral failure — it is a natural consequence of code evolving. Even careful developers make small changes without updating the doc, especially under time pressure. The system is always changing; the spec is static unless someone updates it. The default state is drift. In fact: Drift is inevitable without deliberate effort. The antidote is not care — it is process: traceability matrices, automated tests, and regular reconciliation.

**A stale spec is better than no spec** — At least we have a document, even if it is out of date.

A stale spec is worse than none because people trust it. They make decisions, write tests, and plan features based on it. When it is wrong, those decisions are wrong. A missing spec forces you to look at the code; a stale spec points you the wrong way. In fact: An inaccurate document is a liability. It is better to have no spec and rely on code review than to have one that misleads. If you keep a spec, keep it honest or delete it.

**Once written, the spec is frozen** — The spec is the law — we must never change it.

Requirements change. New information arrives, user feedback comes in, the market shifts. Freezing the spec forces drift because the code will change anyway, and the spec becomes the only thing that does not. The spec must be a living document. In fact: The spec should be versioned and updated through a change process. When code changes for a legitimate reason, update the spec deliberately. The goal is alignment, not rigidity.

**Spec drift only matters for documentation** — Spec drift is just a documentation problem — the code still works, so why bother?

The code might work, but the spec is the shared understanding. When a new developer joins, or you return to the project after three months, you will rely on the spec. If it is wrong, you will waste time, introduce bugs, and possibly break something that was working. In fact: The spec is the guide for future work. Drift makes that guide unreliable, leading to wasted effort and avoidable bugs.


## In short

Spec drift is the silent divergence between the written specification and the actual code. It is invisible without traceability, but it is measurable: count what the spec promises, count what the code delivers, and the gap is the drift. A stale spec is worse than none because it is trusted. Regular reconciliation — through tests, traceability matrices, and review gates — is far cheaper than a rewrite later.


## Where this is used

Spec drift is not an abstract worry — it is the reason AI projects fail in production. Here is how it shows up across AI work, and what a developer actually does about it.

**Large language models (LLMs).** When you build a system around an LLM, the specification usually describes the prompt behaviour: 'The assistant must never reveal the system prompt' or 'It must respond in under 100 tokens.' The implementation is the prompt plus the model. As you tweak the prompt to fix one issue, you might break another. A developer will write a test suite that sends specific queries and checks the response. If the spec says 'must say I don't know when asked about medical advice', and the model starts giving advice after a prompt change, that is drift. The spec is still on the page, but the code (the prompt) has moved. To catch it, the developer runs the tests regularly — a practice called regression testing. The test is the traceability link: it ties the spec requirement to a concrete check.

**Small language models on devices (TinyML, mobile LLMs).** Here the constraints are hard: size, memory, and latency. The spec might say 'the model must be under 50 MB and infer in under 200 ms on a Pixel 7.' As you quantise the model or prune layers, you might lose accuracy. Drift here is quantitative: the spec promises a certain accuracy on a benchmark, and the deployed model does not meet it. The developer keeps a performance report that logs accuracy after every change. If the report shows the number dropping below the spec threshold, drift has occurred. The traceability is a simple spreadsheet or a CI job that runs the benchmark and posts the number.

**Retrieval-Augmented Generation (RAG).** RAG systems add a retrieval step to an LLM. The spec might say 'the system should retrieve only from approved knowledge sources' or 'the answer must cite the source.' Drift happens when the embedding model is updated, or the vector database gets stale, so retrieval starts pulling from unapproved sources, or answering without a citation. A developer writes tests that feed a known query and check that the retrieved context is from the allowed list. The traceability is the test, and running it after every embedding model change catches drift early.

**Agentic systems (planning, tool calling, loops).** Agents follow a spec that describes their loop: 'call the weather API, if it fails, retry twice, then escalate.' Drift is a constant threat here because agents change behaviour with the model or the prompt. The spec says one thing, but the agent might start retrying endlessly, or hallucinate a tool call. A developer will write unit tests for the planning logic and integration tests that mock the API and check the agent's calls. Each test is a traceability link to the requirement.

**Everyday applied ML (recommendation, forecasting, fraud detection, speech).** In classic ML, spec drift often means the model is re-trained on new data, and the new model does not behave exactly like the old one. The spec says 'the fraud model should flag transactions above a certain risk score.' After retraining, the score distribution shifts, and fewer transactions are flagged. The developer monitors the model's behaviour and compares it to the spec. Tools like Great Expectations or custom drift detectors can alert when the actual distribution differs from what the spec assumed. This is not just about documentation — it is about model governance.

**What a developer actually writes to control drift.** Beyond the simple script above, they use a traceability matrix — a table with columns for Requirement ID, Test ID, and Code Module. Each requirement links to a test. They run the tests in a continuous integration (CI) pipeline. Whenever the code changes, the tests run. If a test fails, drift is detected immediately. They also write acceptance criteria — concrete, testable conditions — so 'the response should be helpful' becomes 'the response must include a source link when asked for references.' Finally, they review the spec and the code together at regular intervals, like a code review but comparing against the document. When drift is found, the policy is: either fix the code to match the spec, or update the spec to reflect the change — but never leave them in disagreement. Choosing to update the spec is called a 'spec change' and goes through a formal process, so the change is intentional, not accidental.


---

Read this on the web: https://basavapurushottam.com/learn-ai/spec-drift
