# Point-in-Time Data

Reconstructing exactly what was known at a given moment, so models are never trained on information from the future.

**Stands alone:** nothing in the curriculum requires this first, and nothing yet builds on it.

*Data Engineering and Analytics · Data Engineering*


## Why learn this

You trust the news you read each morning to make decisions about your day. You do not want a financial advisor telling you they predicted next week’s stock prices using yesterday’s closing price, nor a weather app blaming its inaccurate forecast on data released after the storm passed. Point-in-time data is the discipline of ensuring that any analysis or decision looks only at the information available at that specific moment, never at information that was discovered later. Without this, every prediction is haunted by hindsight bias — knowing the outcome before it happened. This skill separates trustworthy systems from those that look accurate in testing but fail in reality because they secretly peeked into the future.


## The intuition

Imagine you are a detective trying to solve a case. You have a stack of documents — news articles, police reports, financial statements. Each document contains facts, but here's the catch: each fact was learned at a specific moment. A newspaper published on Monday says the suspect was seen at a café. On Tuesday, a correction says it was actually a different person. If you read the Tuesday correction and then assume the detective on Monday knew about it, your entire reconstruction of the case is wrong. You would blame the wrong suspect. Point-in-time data is the discipline of reconstructing exactly what was known at a given moment, not what we know now. It's about separating the timeline of events from the timeline of when facts were discovered. In finance, this is brutal: a company's revenue for the first quarter might be reported in April, restated in July, and adjusted again in December. A model trained on the December version will fail in production because live it only sees the April version. Point-in-time data is the practice of storing and using the version of a fact that was available at the decision moment, not the latest version. It's the difference between knowing the truth now and knowing what the truth was then.


## An analogy

Think of a library that keeps every edition of every newspaper, not just today's. A journalist writing a story about last year's election cannot quote from an interview that happened after the election — that would be a lie. They must consult the papers as they were printed on each day. The library has a special request form: 'Give me the news as it appeared on March 14th.' That form is point-in-time discipline. The library also keeps every correction and retraction — the March 14th paper might have a footnote on page 2 from March 16th correcting a typo, but the journalist only sees the original text. Point-in-time data is that footnote-free view. A real library has a single physical stack, and retrieving a specific date is straightforward; a real data system must manage this view across multiple sources where updates arrive late or without notice.


## Definition

Point-in-time data is the practice of storing and retrieving facts along with the timestamp of when they were known, so that any analysis or model uses only the information that was available at the moment of the event being studied, never information from the future.


## How it connects

This concept sits in Data Engineering, which covers the systems that move, shape, and serve data. It relies on Data Modeling, a method for designing how information is structured, because you need to mark each record with when it became valid (valid_from) and when it was replaced (valid_to). It connects to SQL, the standard language for querying databases, because you will filter records by asking which versions were active on a specific past date. It also relates to Entity Resolution, the process of determining whether records from different sources refer to the same real-world object, because tracking changes over time requires identifying that same object consistently.


## The mathematics

Point-in-Time data is a rule for keeping time straight when you are looking at history. It stops you from accidentally using information that did not exist yet at the moment you are trying to understand.

Imagine you are standing on June 1st. You want to know what the company's earnings were *at that moment*. You look at your records. You see a report dated April 15th saying the earnings were 100. You also see a later report dated July 20th saying the earnings were actually 90.

On June 1st, you do not know about the July 20th correction. That information is in your future. If you use the number 90, you are cheating. You are using knowledge that was not available to anyone on June 1st. This mistake is called 'look-ahead bias'.

The math here is just a comparison of two dates.

Let us name the moment we are analyzing. We call this the Event Date. Let us write it as t_event.
Let us name the date when a piece of information became known to the public. We call this the Fact Date. Let us write it as t_fact.

The rule is simple:
A fact is allowed to be used for an event only if the fact was known on or before the event happened.

In symbols, we write:
t_fact <= t_event

This reads: 'the time the fact became known is less than or equal to the time of the event.'

If this statement is true, the fact is 'admissible'. It is safe to use.
If this statement is false (meaning t_fact is greater than t_event), the fact is from the future. It must be thrown away.

There is no complex formula. There is no probability. There is no summation. There is only this single check: Did I know this before I made the decision?

In a database query, this looks like:
WHERE t_fact <= t_event

This ensures that every piece of data used in your analysis was actually available at the time the event occurred.


## A worked example

Let us walk through a specific case to see how this works.

Scenario: We are analyzing a stock price on June 1st. We want to use the company's quarterly earnings as one of the inputs for our analysis.

Step 1: Identify the Event Date.
The moment we are analyzing is June 1st. So, t_event = June 1st.

Step 2: Identify the available facts and their dates.
Fact A: The company reported Q1 earnings of 100 units on April 15th.
So for Fact A, t_fact_A = April 15th.

Fact B: The company restated Q1 earnings to 90 units on July 20th (due to an error found later).
So for Fact B, t_fact_B = July 20th.

Step 3: Check Fact A against the Event Date.
We ask: Was Fact A known on or before June 1st?
Is April 15th <= June 1st?
Yes. April comes before June.
Conclusion: Fact A is admissible. On June 1st, the world knew the earnings were 100. We must use 100.

Step 4: Check Fact B against the Event Date.
We ask: Was Fact B known on or before June 1st?
Is July 20th <= June 1st?
No. July comes after June.
Conclusion: Fact B is NOT admissible. On June 1st, no one knew about the restatement yet. The information did not exist in the public domain. We must ignore it.

Step 5: Determine the value to use.
On June 1st, the only valid earnings figure is 100.
If we had used 90, we would have been using information from July (the future) while pretending to be in June (the past). This invalidates any test of our model because real traders on June 1st could not have seen that number.

Result: The correct Point-in-Time value for earnings on June 1st is 100.


## From scratch, in Python

```python
# Point-in-time filtering in plain Python

# A fact is a tuple: (value, known_date)
# A tuple is like a list but cannot be changed after creation.
# We'll use a list of such tuples.

facts = [
    (100, '2024-04-15'),  # Q1 earnings as originally reported
    (90,  '2024-07-20'),  # restatement
]

event_date = '2024-06-01'

# Function to filter facts known on or before the event date
# A function is a reusable block of code. It takes an input and returns an output.

def get_known_facts(facts_list, as_of_date):
    """Return facts whose known_date is on or before as_of_date."""
    known = []
    for value, date in facts_list:
        # Compare dates as strings (YYYY-MM-DD sorts chronologically)
        if date <= as_of_date:
            known.append((value, date))
    return known

# Call the function
usable = get_known_facts(facts, event_date)
print(usable)  # Output: [(100, '2024-04-15')]

# The latest usable value:
latest_value = usable[-1][0]  # -1 gets the last element, [0] gets the value
print(latest_value)  # Output: 100
```


## The same thing, with a framework

```python
In real systems, you would use a database or a DataFrame library like Pandas or Polars. Here's the Pandas equivalent: ```python
import pandas as pd

facts_df = pd.DataFrame({
    'value': [100, 90],
    'known_date': ['2024-04-15', '2024-07-20']
})
event_date = '2024-06-01'

usable = facts_df[facts_df['known_date'] <= event_date]
print(usable)
```  Pandas does the same comparison internally — it is not magic. The library lets you avoid writing the loop yourself, but the logic is identical.
```


## What the code does

Line by line: We create a list `facts` containing two tuples. Each tuple has a value and a date string. We set `event_date`. The function `get_known_facts` takes a list and a date. It initializes an empty list `known`. It loops over each `(value, date)` in the input list. The `if` compares the date strings — because they are in YYYY-MM-DD format, string comparison works as date comparison. If the date is earlier or equal, it appends the tuple to `known`. After the loop, it returns `known`. We then call it and print, seeing only the April fact. The last line shows how to get the latest usable value. The key idea: we are filtering by `date <= as_of_date`, exactly the formula.


## Common misconceptions

**Restated data is better data, so use it** — The most recent restatement is the most accurate, so I should always use the latest version of a fact.

The latest version includes corrections that were not known at the time of the event. Using it injects future knowledge into the analysis, which invalidates any backtest or model training. In fact: For any analysis, you must use the fact as it was known at the analysis date. The latest version is only correct if the analysis date is after the restatement timestamp.

**Point-in-time is only for finance** — This seems like an accounting trick, not relevant to my ML projects.

Any domain with time-stamped events — e-commerce, healthcare, fraud, IoT — has the same risk of future leakage. The mechanism is identical. In fact: Point-in-time discipline applies everywhere that data has a discovery time, not just finance.

**Timestamps are optional metadata** — I can add timestamps later if I need them.

If you do not record when a fact was known at ingestion, you can never recover it. The reconstruction requires the original timestamp. In fact: Point-in-time discipline starts at the point of data capture — you must store the 'as of' timestamp from the beginning.

**Storing the latest version is enough** — I keep the main table with all the values, and I just use the latest one when training.

You lose the historical revisions, making it impossible to know what was known at any past moment. In fact: You need a history table that stores every version along with its valid_from and valid_to timestamps.

**If the backtest looks good, it's fine** — My backtest accuracy is high, so the model is good.

A high backtest accuracy could be a symptom of future leakage, not genuine predictive power. It will crash in production. In fact: You must validate that no future information was used, either by auditing the pipeline or by testing on a held-out set with point-in-time filtering.


## In short

Point-in-time data means using only the facts that were known at the event time, not the latest facts. It requires storing timestamps of when each fact was known, and filtering on `known_date <= event_date`. Getting it wrong silently invalidates any backtest or model. This is the foundation for trustworthy data engineering and ML.


## Where this is used

Point-in-time data is everywhere in applied machine learning, and getting it wrong is the most common silent killer of backtests.

**Large Language Models (LLMs)** — An LLM's training data is a snapshot of the web at a point in time. When you fine-tune an LLM on financial documents, you must ensure that the documents used for training have a timestamp earlier than the test period. For example, if you are building an earnings call assistant, you cannot train on a transcript that was published after the call date — that would leak the answer. In practice, developers store each document with a `published_at` timestamp and filter training data to `published_at < event_date`. The book *Large Language Models in Finance* stresses this: using information the model could not have had at the time silently invalidates any backtest. When an LLM is used for sentiment analysis of a stock on June 1st, the model must not have been trained on June 2nd news.

**Small Language Models (SLMs)** — On-device models, like a keyword spotter on a phone, are trained once and deployed. The training data must be point-in-time relative to deployment. If you collect user interactions and a week later the schema changes, you must not use the new features in training for older events. Developers maintain a versioned feature store where each feature's value has a `valid_from` timestamp, and training data is generated by joining events to features as of the event time.

**Retrieval-Augmented Generation (RAG)** — RAG systems retrieve documents at inference time to ground an LLM's answer. If the retrieval index does not filter by point-in-time, a question about Q1 earnings might retrieve a restated document from July, and the model would answer with 90 instead of 100. The fix is to store each chunk of text with a `document_date` and, at query time, add a filter `document_date <= query_date`. Developers using vector databases like Pinecone or Weaviate must include this filter in the metadata. A developer writes: `index.query(vector, filter={'date': {'$lte': '2024-06-01'}})`.

**Agentic Systems** — Agents that plan and call tools to make decisions (e.g., a trading agent) must never see future facts. If an agent is given a current date and a tool that returns the latest news, it might access a news article from tomorrow (if the data pipeline accidentally includes future news). Point-in-time discipline requires that every tool's output includes a `generation_timestamp` and that the agent's planner filters out anything with a timestamp beyond its current position. Developers must carefully pass the current time into each tool and validate the returned data's timestamps.

**Everyday Applied ML** — Recommendation systems, fraud detection, forecasting — all of them. A fraud model trained on transaction data must use only the features that were computable at the time of the transaction, not features derived from later transactions (like 'is_fraud' after a chargeback is filed). A forecasting model for product demand must use only sales data up to the forecast date, not the actual demand that shows up later. Developers use feature stores like Feast or Tecton, where each feature is defined with a point-in-time join. In practice, a developer writes a SQL query like `SELECT * FROM transactions t LEFT JOIN features f ON t.customer_id = f.customer_id AND f.as_of_date <= t.timestamp`. If this join is missing the `as_of_date` condition, the model silently learns from the future. The break when done badly is insidious: the backtest looks amazing, but live performance craters because the future information is gone.


---

Read this on the web: https://basavapurushottam.com/learn-ai/point-in-time-data
