# Vector Native Storage

Storage designed around embeddings as a first-class type, with indexes, filters and hybrid scoring built in rather than bolted on.

**Rests on:** Vector Spaces and Rank

**Leads to:** Vector Search

*LLMs and Generative AI · Retrieval-Augmented Generation*


## Why learn this

You need this because you want to build systems that answer questions using your own private documents instead of relying solely on a model's general training data. This capability is called Retrieval-Augmented Generation, often shortened to RAG. In a RAG system, when someone asks a question, the system first needs to find relevant information from your document collection. Without understanding how this search works internally, you cannot choose the right technology to store that information, debug why it returns irrelevant results, or design the necessary filters (such as limiting answers to documents published after 2023). This topic directly explains how 'Vector Search'—the method of finding related content by meaning rather than exact word match—operates under the hood, and how it can be combined with traditional keyword matching for better accuracy.


## The intuition

Think of a library. A traditional database is like a librarian who only knows the exact titles of books. If you ask for 'a book about the history of railways in India', they can't help unless someone has catalogued that exact phrase in the title or author field. A vector-native database is like a librarian who understands meaning. You can say 'railway history India' and they bring you the closest books, even if those specific words do not appear in the text. This is possible because we convert text, images, or any data into a list of numbers called an embedding—a unique numerical fingerprint that captures the essence of the content. The database stores these embeddings as a primary feature, meaning it is designed from the ground up to store, index, and search these fingerprints quickly, even when you have millions of them. It also lets you combine this meaning-based 'similarity search' with other filters—like 'only books published after 2000' (known as metadata filtering)—and even mix in keyword search for exact terms. The key insight is that a regular database, which organises data in rigid tables and rows, is built for exact lookups and sorting. It fails when asked to find 'close' matches in this high-dimensional space of numbers. Therefore, the storage system itself must change to handle these numerical fingerprints natively.


## An analogy

A library card catalogue. Old libraries had a physical card catalogue with one card per book, filed strictly alphabetically by title, author, or subject. That works if you know the exact title or author. But what if you want 'a book that reminds me of a summer afternoon in a small town'? The old catalogue is useless because it only orders by simple, rigid keys like name or date. Now imagine a new system: every book gets a card with a 'taste fingerprint'—a list of numbers describing its themes, writing style, and emotional tone. When you come in and describe your summer afternoon, the librarian converts your description into a similar fingerprint and then walks through the library, pulling out books whose fingerprints are closest to yours. That is a vector-native library. It can also handle filters: 'only books under 300 pages'—the librarian skips ones that don't fit that page count while still looking for close fingerprints. And it can combine with the old keyword search: 'I specifically want a book with the word monsoon in the title, but also with a similar feel.' The analogy breaks down because the librarian cannot literally scan millions of books with every query—that is where pre-computed indexes come in, acting like a shortcut map of which fingerprints are near each other. And the fingerprint isn't perfect—it captures one aspect of meaning, not the whole book. But the core idea holds: store a numerical fingerprint alongside the data, and make finding similar fingerprints the database's superpower.


## Definition

Vector-native storage is a database system where the numerical representation of meaning (the embedding) is a first-class citizen, supported by special indexes for fast similarity search, built-in capability to filter by auxiliary information (metadata), and native ability to combine meaning-based scoring with traditional keyword scoring as core features rather than afterthoughts.


## How it connects

This sits at the heart of Retrieval-Augmented Generation (RAG), which is the process of retrieving relevant documents to ground an AI's answer. In RAG, you ingest documents, split them into chunks, and create an embedding for each chunk. When a query arrives, you use this storage to find the most similar chunks via 'Vector Search'—the efficient method of locating related items in a vast numerical space without checking every single one individually. This also enables Hybrid Search, which combines vector similarity with traditional keyword matching to balance precision and recall. Your notes mention that combining hybrid search with 'reranking' (a second pass that re-evaluates and orders the retrieved results for final relevance) beats pure vector similarity in production settings—this is only possible if the storage supports both kinds of scoring natively. It also connects to Citation, because you need to retrieve and cite specific passages, meaning the storage must return not just a chunk but its source location.


## The mathematics

The mathematics behind vector-native storage rests on two simple ideas: representing information as lists of numbers, and measuring how close those lists are to each other.

First, we represent every piece of data (like a document or an image) as a list of numbers. Mathematicians call this a "vector." For our purposes, think of it simply as an ordered list. If the list has three numbers, we can picture it as a point in a 3D space, but the same logic works for lists with hundreds or thousands of numbers.

To compare two pieces of data, we need a way to measure their similarity. We do this using an operation called the "dot product." The dot product takes two lists of the same length and produces a single number that tells us how aligned they are.

Here is exactly how you calculate the dot product:
1. Take the first number in the first list and multiply it by the first number in the second list.
2. Take the second number in the first list and multiply it by the second number in the second list.
3. Continue this for every position in the lists.
4. Add all those products together.

The result is a single score. If the two lists have many large numbers in the same positions (meaning they point in a similar direction), the sum will be a large positive number. This indicates high similarity. If one list has large positive numbers where the other has large negative numbers, the products will be negative, and the sum will be low or negative, indicating they are very different.

In vector-native storage, we use this score to find relevant information. When you ask a question, we turn it into a query list (the query vector). We then calculate the dot product between this query list and every stored list in our database. The lists that produce the highest scores are the ones most similar to your question.

Because calculating this for millions of lists is slow, we use special structures (like trees or graphs) to group similar lists together. This allows us to skip lists that are clearly far away from the query and only calculate the dot product for the most likely candidates.


## A worked example

Let's walk through a concrete example with small lists of three numbers.

We have:
- A query list (your question): q = [0.9, 0.1, -0.4]
- Stored item A: a = [0.8, 0.2, -0.5]
- Stored item B: b = [-0.3, 0.7, 0.6]

We want to know which item, A or B, is more similar to the query q.

Step 1: Calculate the similarity score for q and a (the dot product of q and a).
- Multiply the first numbers: 0.9 * 0.8 = 0.72
- Multiply the second numbers: 0.1 * 0.2 = 0.02
- Multiply the third numbers: -0.4 * -0.5 = 0.20 (Note: a negative times a negative is positive)
- Add these results together: 0.72 + 0.02 + 0.20 = 0.94

So, the similarity score between q and a is 0.94.

Step 2: Calculate the similarity score for q and b (the dot product of q and b).
- Multiply the first numbers: 0.9 * -0.3 = -0.27
- Multiply the second numbers: 0.1 * 0.7 = 0.07
- Multiply the third numbers: -0.4 * 0.6 = -0.24
- Add these results together: -0.27 + 0.07 + (-0.24) = -0.44

So, the similarity score between q and b is -0.44.

Step 3: Compare the scores.
- Score for a: 0.94
- Score for b: -0.44

Since 0.94 is much larger than -0.44, item A is more similar to the query q than item B is.

In a real system, we would do this calculation for every stored item and return the ones with the highest scores.


## From scratch, in Python

```python
import math

# Representing embeddings as lists of floating-point numbers
query = [0.9, 0.1, -0.4]
stored_a = [0.8, 0.2, -0.5]
stored_b = [-0.3, 0.7, 0.6]

# Function to compute dot product
def dot(a, b):
    total = 0.0
    for i in range(len(a)):
        total += a[i] * b[i]
    return total

# Compute scores
score_a = dot(query, stored_a)
score_b = dot(query, stored_b)
print("Score for a:", score_a)
print("Score for b:", score_b)

# Ranking
if score_a > score_b:
    print("a is more similar")
else:
    print("b is more similar")

# Adding a metadata filter: only consider items with a 'year' >= 2020
# We'll use a simple list of dictionaries
items = [
    {'embedding': stored_a, 'year': 2021, 'id': 1},
    {'embedding': stored_b, 'year': 2019, 'id': 2},
]

filtered_scores = []
for item in items:
    if item['year'] >= 2020:
        score = dot(query, item['embedding'])
        filtered_scores.append((score, item['id']))
print("Filtered scores:", filtered_scores)
```


## The same thing, with a framework

A developer would rarely write this by hand. They'd use a library like FAISS or a database like ChromaDB, pgvector, or Weaviate. For example, in Python with ChromaDB: 

```python
import chromadb
client = chromadb.Client()
collection = client.create_collection('docs')

# Add documents with their embeddings
collection.add(
    ids=['1', '2'],
    embeddings=[[0.8, 0.2, -0.5], [-0.3, 0.7, 0.6]],
    metadatas=[{'year': 2021}, {'year': 2019}],
    documents=['doc one', 'doc two']
)

# Query
results = collection.query(
    query_embeddings=[[0.9, 0.1, -0.4]],
    n_results=2,
    where={'year': {'$gte': 2020}}
)
print(results)
```

This library call does the same as our plain Python: it computes dot products (or other similarities) and applies the metadata filter, but it does so using an optimised index underneath, so it's fast even with millions of vectors.


## What the code does

The code first defines two embeddings and a query as Python lists. Then it defines a function `dot` that takes two lists, loops through each index, multiplies corresponding elements, and sums them — exactly the dot product we did by hand. It applies this to get similarity scores for both stored vectors and prints which is higher. Then it shows a more realistic scenario: the stored items are not just bare lists, but dictionaries with extra fields like 'year'. A dictionary is a way to store named values, accessed by key. The code loops through a list of these dictionaries, uses an `if` statement to check the year against a threshold, and only computes the dot product for those that pass. This mimics metadata filtering. In a real vector database, you'd have millions of such items, and the database handles indexing and filtering internally for speed.


## Common misconceptions

**Embeddings are just compression** — An embedding is just a way to shrink a large document into a small number, losing some detail but keeping the gist.

Compression aims to reconstruct the original as closely as possible. Embeddings are trained so that distances in the embedding space correspond to semantic similarity — they are not reversible to the original text, and they capture a specific, learned notion of meaning. Two different compression algorithms would give similar reconstructions; two different embedding models give very different embeddings. In fact: An embedding is a learned representation that places similar items close together in a high-dimensional space. It's a mapping, not a compression.

**Vector databases are just tables with a special column** — A vector database is a regular SQL database where you add a column of type 'vector' and it automatically works.

A regular database stores rows and uses B-tree indexes for exact lookups. A vector column wouldn't be indexed, so every query would be a full table scan, which is extremely slow for millions of rows. True vector databases use specialised index structures (like HNSW graphs or IVF inverted files) that are built specifically for similarity search and support approximate searches that can skip large portions of the data. In fact: A vector-native database has a fundamentally different storage and indexing architecture, designed to efficiently find items by similarity in high-dimensional space, not just by exact value.

**The dot product is the only similarity measure** — The way to compare embeddings is always to compute the dot product.

Different embeddings are trained with different distance measures in mind. Common ones include cosine similarity (which normalises for the length of the vectors), Euclidean distance, and dot product. If the embeddings are normalised to unit length, they are equivalent, but that's not always the case. Using the wrong measure can give poor results. In fact: The choice of similarity metric (cosine, dot product, Euclidean distance) is a design decision that should match how the embedding model was trained.

**Filters can always be applied after retrieving the top-N** — You can just retrieve the top 100 similar items, then filter out those that don't match the metadata, and you'll still get good results.

If the metadata filter is very selective (e.g., only 1% of items match), the top 100 results might all be filtered out, and you'll end up with zero or very few items. The database must apply the filter before or during the similarity search to find the most similar items *among those that pass the filter*. In fact: Vector databases integrate metadata filtering into the index, so they can find the nearest neighbours that also satisfy the constraints, ensuring you get the best matches within the allowed set.

**Vector-native storage is only about speed, not quality** — Using a vector database just makes retrieval faster; it doesn't change which documents you get back compared to scanning everything.

The index structure and the choice of distance metric affect which items are considered 'nearest'. An approximate index might return slightly different results than a exhaustive search, and the order of index construction (how vectors are grouped) influences the outcome. The storage also decides the top-k results, which directly determines what goes into the LLM's context and therefore the quality of the answer. In fact: Vector-native storage influences both retrieval speed and the set of candidates considered, and its design choices (index type, metric, filter integration) have a direct impact on the relevance of the results.


## In short

Vector-native storage is a database built around embeddings, using specialised indexes for fast similarity search, native metadata filters, and hybrid scoring. It is the foundation of RAG and semantic search. You store each chunk as a vector, search by dot product or cosine similarity, and combine with filters to get precise, relevant results. Remember: the index matters more than you think, filters should be applied during search, and the similarity metric must match the embedding model.


## Where this is used

Vector-native storage is the backbone of any modern search system that uses embeddings. Here's how it's used across AI:

**Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG):** When you use a service like ChatGPT with a custom knowledge base, the flow is: you upload documents → the system chunks them → embeds each chunk into a vector → stores them in a vector-native database. When you ask a question, it embeds the question and searches the database for the closest chunks. These chunks are then placed in the LLM's context window, so the model can formulate an answer grounded in your documents. In practice, a developer would define a vector collection with fields like 'text' and 'source', and then query it with `collection.query(query_embeddings=[...])`. They might add filter options like `where={'category': 'manual'}` to restrict to a specific subset. The database's speed is critical: some production RAG systems retrieve from billions of chunks in milliseconds.

**Small Language Models (on-device):** For on-device models (like a phone assistant that runs locally), the embedded index also runs locally. Here, vector-native storage is often embedded directly in the app, using lightweight libraries like SQLite with the `sqlite-vec` extension or a local FAISS index. The developer writes a small query function that takes a voice prompt, converts it to an embedding (using a small model), and searches the on-device index. The constraint is memory: the index must fit in the device's RAM, so these systems use quantized embeddings and aggressive compression. The same concepts apply: similarity search plus metadata filters, just on a smaller scale.

**Retrieval-Augmented Generation (RAG) pipelines in enterprises:** Here, vector-native storage must handle access control. For example, in a legal firm, only certain lawyers should retrieve client documents. The storage must filter by metadata like `client_id` and `security_level` during the search, not after, to prevent leaking data. This is what your library notes call 'permission-aware retrieval'. A developer would write queries with `where={'client_id': <id>}` and possibly use per-user index sharding. If the storage doesn't support these filters natively, developers have to manually post-filter results, which is slow and error-prone.

**Agentic systems:** Agents that plan and call tools often need to retrieve information repeatedly and in context. An agent might use vector-native storage to recall past actions or relevant knowledge. For example, an agent planning a trip might query a vector database for 'restaurants near the harbour' and then, based on the results, query again with added filters for 'vegetarian'. The storage's ability to combine filters with similarity quickly is crucial because agents may make dozens of queries per task.

**Everyday applied ML (recommendation, vision, fraud, speech):** In recommendation systems, you embed users and items into the same space, and the storage finds similar items for a user. In vision, you embed product images and search for similar-looking items. In fraud detection, you embed transaction patterns and search for past anomalies. In speech, you embed audio snippets to find similar recordings. In all these cases, the developer applies the same pattern: define an embedding function, store the vectors in a vector-native database, and query with a combination of similarity and filters (e.g., filter by category, date, or risk score). The library they'd reach for is either a standalone vector DB like Milvus, Pinecone, or Weaviate, or an extension to a relational DB like `pgvector` in PostgreSQL.

**What breaks when done badly:** If the storage doesn't have a proper vector index, every query becomes a full scan — comparing the query against every single stored vector. With a million vectors, that can take minutes or hours. If filters aren't integrated into the index, the database may retrieve many results then filter them out, wasting compute and potentially missing relevant ones if the top-N are all filtered away. If hybrid scoring isn't supported, you can't combine keyword and semantic search, which your library notes say is 'better than pure vector similarity in most production settings'. If the storage can't handle high-dimensional embeddings (e.g., 1536 dimensions from OpenAI's model), the index structure may degrade, and similarity calculations become numerically unstable. So the choice of vector-native storage is not just an engineering detail — it directly impacts the quality and speed of AI features.


---

Read this on the web: https://basavapurushottam.com/learn-ai/vector-native-storage
