# Cross-Modal Drift

Producing output plausible for the training distribution generally, but not for this particular input.

**Rests on:** Multimodal Grounding Failures

*LLMs and Generative AI · Hallucination Mitigation*


## Why learn this

Cross-modal drift is the reason you cannot trust a description of an image just because it sounds fluent. It is the difference between a system that merely reads well and one that actually sees. Understanding it gives you the ability to audit any multimodal AI — a captioning system, a visual assistant, an agent that acts on what it 'sees' — and to build systems that verify what they claim. It unlocks further study in hallucination mitigation, multimodal grounding, and evaluation, which you already have notes on: Object Hallucination, Multimodal Grounding Failures, and Content Moderation Guardrails.


## The intuition

Imagine you ask a friend to describe a photo they have not seen. They would produce a description that sounds entirely plausible — 'a sunny park with children playing' — but it is not about the actual photo. Cross-modal drift is exactly that: a model generates output that is consistent with the kind of input it has seen before (the training distribution) but not with the specific input you gave it. It is subtle because nothing about the output looks wrong. The words are correct, the grammar is perfect, the details are ordinary. Only a fact-check against the input reveals the mismatch. In multimodal systems, the input is typically an image plus a text prompt. The model's job is to describe what is in the image. But the model has learned patterns from millions of images and captions, and when the image is ambiguous or the language priors are strong, the model falls back on what it has seen before, not what is in front of it. The output drifts away from the true contents of the image, even though it drifts toward what is statistically likely. That is the drift: the gap between what the model says and what the input actually contains.


## An analogy

Think of a tour guide who has memorised a script about a city but is asked to give a tour of a specific house. The guide knows the city's general layout, the typical architecture, the common features of houses in that area. But the house they are shown has a peculiar feature — say, a bright purple door. The guide, relying on the script, says 'and here you see the standard oak door', because that is what is typical. The description is plausible for the city, but it is wrong for this house. The guide has drifted from the actual house to the general pattern. That is cross-modal drift. The model does not have a script in the same way, but it has learned statistical associations. When the image contradicts the statistical norm, the norm often wins. The analogy works well because it captures the core: fluency without faithfulness. It breaks down because the tour guide could, if asked, look again and correct themselves. A neural network cannot easily 'look again' — it processes the input once, and its output is already determined. It also cannot reason about what it sees; it just generates text that is probable given the input. So the guide analogy understates how persistent the drift is. The model will confidently assert the wrong thing without any awareness of the error.


## Definition

Cross-modal drift is the production of output that is plausible for the training distribution as a whole but not for the specific input provided, because the model relies on learned language priors rather than the actual non-text evidence.


## How it connects

You have notes on hallucination mitigation, and cross-modal drift is one of its failure modes in multimodal systems. It is closely related to two neighbouring topics you already have notes on: Object Hallucination (the model describes entities not in the image) and Multimodal Grounding Failures (the broader class of outputs making claims the non-text evidence does not support). Cross-modal drift is distinct from object hallucination in that it is not limited to missing objects — it can be about attributes, relationships, spatial layout, or any detail. It is also related to content moderation guardrails, because a guardrail that checks whether the output is grounded in the input is a way to catch drift.


## The mathematics

This concept relies on how a computer model makes choices using numbers called probabilities. A probability is a number between 0 and 1 that tells you how likely something is to happen. A value of 1 means it will definitely happen; a value of 0 means it will never happen.

The model works by looking at an input, which we will call x. The input could be an image or a piece of text. The model then looks at every possible answer, which we will call y. For each possible answer y, the model calculates a number P(y | x). This is read as "the probability of y given x." It tells you how likely the model thinks that specific answer is, based on the input it sees.

The model does not guess randomly. It follows a simple rule: it picks the answer with the highest probability. We write this choice as ŷ (read as "y-hat"). The formula for this choice is:

ŷ = the y that makes P(y | x) as large as possible.

In plain words, the model scans all its options and selects the one with the biggest number attached to it. This works well on average because the model was trained on many examples where the correct answers were known. The training process adjusted the model's internal numbers so that, for most inputs, the highest probability matched the truth.

Cross-modal drift happens when this rule fails for a specific case. It occurs when the answer with the highest probability (ŷ) is wrong, even though the number P(ŷ | x) is large. This means the model is very confident in its choice, but that confidence is misplaced. The source of this error is the gap between what the model learned (its own distribution of probabilities) and what is actually true in the real world (the true conditional distribution). The model has learned patterns that do not always hold for every single new input.


## A worked example

Let us look at a specific case with real numbers to see how this works.

Imagine a model that looks at an image and must choose one word from three options: 'cat', 'dog', or 'elephant'.

Case 1: The input is a clear photo of a cat.
The model calculates the probability for each word:
- For 'cat': P('cat' | x) = 0.7
- For 'dog': P('dog' | x) = 0.2
- For 'elephant': P('elephant' | x) = 0.1

The model compares these numbers: 0.7, 0.2, and 0.1.
The largest number is 0.7.
Therefore, the model chooses ŷ = 'cat'.
Result: The output 'cat' matches the image. The model is correct.

Case 2: The input is a photo of a cat, but it is blurry or taken in poor light (a different x).
The model calculates new probabilities for this specific image:
- For 'cat': P('cat' | x) = 0.4
- For 'dog': P('dog' | x) = 0.5
- For 'elephant': P('elephant' | x) = 0.1

The model compares these new numbers: 0.4, 0.5, and 0.1.
The largest number is now 0.5.
Therefore, the model chooses ŷ = 'dog'.
Result: The output is 'dog', but the image contains a cat. The model is wrong.

Analysis of the drift:
In Case 2, the model assigned a high probability (0.5) to 'dog'. It was confident in its choice because 0.5 was the highest number available. However, the true answer was 'cat'. This mismatch between the model's highest confidence and reality is cross-modal drift. The model followed its rule correctly (pick the highest number), but the numbers it produced were misleading for this specific input.


## From scratch, in Python

Here is a minimal simulation. We have a small vocabulary and a 'model' that outputs probabilities, but it does not know the actual content of the image — it just has learned biases. We compare its output to the ground truth to detect drift.

```python
# We'll simulate a model that has a bias toward common objects.
# For a given image, the model outputs a probability for each word.

# Step 1: Define the vocabulary.
words = ['cat', 'dog', 'elephant', 'tree']

# Step 2: Define a 'model' that always outputs the same probabilities
# regardless of the image. This is a crude simulation of a model that
# relies on language priors only.
def model_output(image_ignored):
    # These are the model's learned priors: 'dog' is considered most likely.
    probs = {'cat': 0.2, 'dog': 0.5, 'elephant': 0.1, 'tree': 0.2}
    return probs

# Step 3: Ground truth for a specific image. The image actually contains a cat.
ground_truth = 'cat'

# Step 4: Get the model's output and pick the word with the highest probability.
probs = model_output(None)  # the image is ignored in this simulation
prediction = max(probs, key=probs.get)

# Step 5: Check if the prediction matches the ground truth.
if prediction == ground_truth:
    print('Grounded: the model is right.')
else:
    print('Drift: predicted', prediction, 'but the image contains', ground_truth)
```

This code teaches you the core: the model has a distribution, we pick the most probable, and we compare it to the ground truth. In real models, the probabilities come from a complex neural network, but the principle is the same.


## The same thing, with a framework

In practice, you would use a library like PyTorch or a multimodal model like CLIP. For example, using the `transformers` library, you might call a model to caption an image, and then use a separate model to check whether the caption mentions objects that are detected in the image. This is what the code above does manually. The framework version would look like:

```python
from transformers import pipeline

captioner = pipeline("image-to-text", model="blip-image-captioning-base")
detector = pipeline("object-detection", model="facebook/detr-resnet-50")

image_path = "cat.jpg"
caption = captioner(image_path)[0]["generated_text"]
detections = detector(image_path)
labels = [d["label"] for d in detections]
if "cat" not in labels and "cat" in caption:
    print("Drift detected: caption mentions cat but no cat detected.")
```

Here the framework does the heavy lifting. The captioner produces text, and the detector finds objects in the image. If the caption mentions an object not found by the detector, that is a candidate for drift. The underlying principle is the same as our plain Python example: compare the model's claim to the evidence.


## What the code does

First we define a list of possible words. A list in Python is an ordered collection that can be changed; here it holds four strings. Then we define a function called `model_output` that takes an argument (the image, which we ignore) and returns a dictionary. A dictionary is a collection of key-value pairs; here the keys are words and the values are probabilities. We set the probabilities high for 'dog' and low for others, simulating a model with a strong prior for 'dog'. Then we define `ground_truth` as 'cat' to represent what the image actually contains. We call `model_output`, get the dictionary, and use the `max` function with the key argument to find the word with the highest probability. In this case, 'dog' has the highest probability (0.5). We compare that prediction to the ground truth. Since they differ, the output is 'Drift: predicted dog but the image contains cat'. This is the simplest possible detection of cross-modal drift: compute the model's output and compare it to a known ground truth. In production, ground truth may be unknown, but you can use other signals like a separate validation model.


## Common misconceptions

**Drift is just a kind of hallucination** — Cross-modal drift is the same thing as hallucination, just for images.

Hallucination is a broader term for any fabricated content. Cross-modal drift is specifically about the *fit* between the output and the provided input. It is a failure of grounding, not just a failure of factuality. The output may be factually plausible in general but wrong for this instance. In fact: Cross-modal drift is a subclass of hallucination, but it is characterised by the output being plausible for the distribution but not for the specific input, and it is particularly relevant in multimodal systems.

**If the output is grammatically correct, it is grounded** — The model would not produce such a fluent and confident description if it were making it up.

The model's fluency is learned from the training distribution. It has no intrinsic connection to the truth of the claim. The model can be perfectly fluent and entirely wrong. In fact: Fluency and grammaticality are uncorrelated with grounding. A grounded output must be verifiable against the input, not just readable.

**The model can 'look again' if asked** — If I ask the model to double-check the image, it will correct itself.

Many models are not autoregressive over the image; they process it once and generate text from a fixed representation. They cannot 'look again' in the way a human can. Even if they have attention, they cannot choose to re-inspect the image unless trained to do so. In fact: To get a model to correct itself, you must change the input or use a separate verification model. The model itself does not have a mechanism for self-correction based on visual re-examination.

**Detection is just a matter of asking the model for confidence** — If the model outputs a low confidence score, we know it is drifting.

The model's confidence is also a learned estimate, and it is often miscalibrated. A model can be highly confident and wrong. In fact: Confidence scores are not reliable indicators of grounding. You need external verification, such as comparing to an object detector's output or a separate model.

**It only happens in models that are bad** — Only poorly trained models hallucinate or drift.

Even state-of-the-art models exhibit cross-modal drift. The root cause is the fundamental gap between the training distribution and any individual input, which no amount of data can completely close. In fact: Drift is a property of probabilistic systems, not a sign of incompetence. All probabilistic models can produce outputs that are unlikely given the true input.


## In short

Cross-modal drift is the failure of a model to produce output that is faithful to the specific input, instead producing output that is plausible for the training distribution. It is subtle, fluent, and dangerous. The core idea is that probability does not equal truth. To detect drift, you must ground the output against the input using independent evidence, not confidence. You must also be aware that your evaluators can drift, so you need to monitor them. Remember: fluency is not factuality.


## Where this is used

Cross-modal drift is a central challenge in any system that combines vision and language.

**In large language models with vision (VLLMs):** Models like GPT-4V or Gemini can take an image and a text prompt and answer questions about the image. Cross-modal drift appears when the model describes objects, colours, or relationships that are not present, or when it answers a question about the image incorrectly even though the answer is 'not visible'. For a developer, this means that when building a visual Q&A system, you cannot trust the fluent answer. You must add a verification step: use a separate object detector to check whether the entities mentioned in the answer actually appear in the image. If not, you can ask the model to abstain or flag the answer as low-confidence. The application is in debugging and in adding guardrails. A real-world example: a medical image analysis system might describe a scan as showing a tumour when the tumour is not actually highlighted in the image. The guardrail would be a deterministic check: if the segmentation model does not output a region for the tumour, the system should not claim it exists.

**In small models on devices:** TinyML models that run on phones or embedded devices often have to be small, so they may have fewer capabilities and more susceptibility to drift. For example, a smart camera that describes scenes for visually impaired users might say 'a person is crossing the street' when the person is actually a shadow. The drift is caused by the model's priors: it has seen many images of people crossing streets, so it is likely to say that even when the visual evidence is ambiguous. For a developer, this means that when deploying on-device, you must test on real-world data from the device's context, and you may need to incorporate a simple verification step — for example, a motion detector to confirm that the object is moving. Cross-modal drift is more dangerous here because the system has less capacity to reason.

**In retrieval-augmented generation (RAG):** RAG combines a text retriever with a language model, but it can also be multimodal, retrieving images or video. Cross-modal drift occurs when the language model, given a retrieved image, describes the image based on its language priors rather than the image's actual content. For example, a RAG system for a news article might retrieve an image of a fire, but the model might describe it as a 'house fire' when the image is actually of a forest fire. The drift is subtle because the general topic is the same. To mitigate, you must ground the generation in the image by using an image captioning model or an object detector, and you should require the final answer to cite specific image regions.

**In agentic systems:** An agent that perceives the world through images and acts on them (e.g., a robot that uses a vision model to decide what to pick up) is particularly vulnerable to cross-modal drift. If the vision model says 'there is a cup on the table' when there is actually a bowl, the agent will make the wrong motion. The drift is not just a text error; it becomes a physical failure. In building such agents, you must include a verification loop: after the agent plans an action based on the described scene, it should re-check the scene with a different sensor or a different model to confirm. The drift can be costly, so robust systems use consensus among multiple models or deterministic checks.

**In everyday applied machine learning:** Recommendation systems that use images (e.g., e-commerce) can drift by recommending a product based on a description that does not match the image. For example, a system sees a blue shirt but recommends it as 'red' because the text metadata says red. This is cross-modal drift between the image and the text of the product. The developer must check consistency between the image and the text, and when they conflict, decide which to trust. In fraud detection, a system that analyses receipts and images might describe a purchase incorrectly, leading to false alarms. In speech, a system that transcribes audio and generates a summary might drift if the summary describes events that were not in the audio.

**In evaluation:** The note from your library about evaluator drift is crucial: evaluators (models that judge quality) also drift. If you use a language model to evaluate whether a caption is grounded, that evaluator itself may rely on priors and accept the fluent caption without checking the image. So you must monitor the evaluator's performance, version it, and test it on adversarial examples. The learner's note is directly relevant: 'Evaluators drift too: the judge model is a dependency that must be versioned and monitored like any other.' This means that when you build a system to detect drift, you must also worry about the drift of the detector itself.

What does building software with this look like? A developer writing a multimodal application would use libraries like `transformers`, `torchvision`, or `openai` to call a vision model. They would write code that sends the image and prompt, receives the output, and then runs a verification function that checks the output against the image content, often using another model or a deterministic algorithm. They might also set up logging to track instances of drift and use that data to improve the guardrails. What breaks when it is done badly: if you do not verify, the system may produce confident nonsense that users trust, which can lead to dangerous outcomes in medical, legal, or physical contexts. The cost of drift is not a metric; it is a real-world error.


---

Read this on the web: https://basavapurushottam.com/learn-ai/cross-modal-drift
