# Enterprise Knowledge Assistants

Assistants grounded in an organisation's own documents, requiring permission-aware retrieval and honest handling of gaps.

**Rests on:** Citations and Evidence, Multi-Document Retrieval

*LLMs and Generative AI · Retrieval-Augmented Generation*


## Why learn this

This concept gets you from 'a chatbot that guesses' to 'an assistant that works on your company’s own documents, with permission checks and honest answers'. Everything later — RAG, agentic retrieval, citations — depends on this foundation. You will build the mental model for how an assistant limits itself to what it may know, and what a developer does when the assistant must say 'I don’t have that'. It is the first step toward a production AI answering system that people can believe.


## The intuition

An enterprise knowledge assistant is like a helpful employee who has read every policy manual, product spec, and support ticket your company owns — but is also given a strict rule: only use information you can point to in those documents. If a question is not covered, you say so, rather than guess. This is different from a general chatbot, which may have memorised facts from the internet and might confidently state something made up. The assistant’s entire job is to be grounded: every answer must trace back to a snippet of text it was given. That means we must first find the right snippets (retrieval), then let the language model write an answer using only those snippets (generation). And because the documents are company secrets, the assistant must respect who is allowed to see what. If you ask about a document you are not allowed to see, the assistant must refuse — it can’t even let a hint slip. The core tension: you want a smart answer, but you want it honest and within bounds. This concept teaches that balance.


## An analogy

Think of a brand-new librarian in a huge, unorganised warehouse. The librarian has never read any book, but has a perfect memory for exactly what is in each book she has been shown on a given day. Every day, people come with questions. Her job: find the book (or books) that might contain the answer, open them to the exact page, read that passage aloud, and then — if her question is about the passage — say that passage. She is not allowed to invent a new page or paraphrase a different book from memory. If she cannot find a relevant page, she must say 'I don’t have information on that.' That is retrieval: the act of finding the right book and passage. Now add a security rule: some books have a red 'confidential' stamp, and only certain people may even look at the cover. The librarian must check the requester’s badge before opening any red-stamped book. Even if the answer is obviously on page 54, she must not say a word if the requester isn’t cleared. That is permission-aware retrieval. The analogy breaks down: a human librarian can skim many books quickly and has a memory of titles and vague topics. An AI assistant does not have a library index — it must search through copies of every document, organised into chunks, and its 'memory' is just the text it is given in the current request. There’s no intuition, no common sense beyond what’s in the prompt. Also, the librarian reads exactly what is on the page; the assistant may paraphrase, but if it paraphrases too loosely it might drift into fabrication — so we must force it to stay close to the passage or give a direct quote. That is why the retrieval step matters: if you give the assistant only the right snippets, the answer stays honest.


## Definition

An enterprise knowledge assistant is a system that answers questions about an organisation’s documents by first retrieving relevant, permission-authorized passages from those documents and then generating an answer grounded only in those passages, explicitly admitting when no relevant evidence exists.


## How it connects

You don’t need any prior knowledge for this page. This is the first step. It sits at the beginning of the Retrieval-Augmented Generation (RAG) family — the technique of adding retrieved documents to a model’s context. You have notes later on: Embeddings (how documents become numbers so we can search by meaning), Vector Databases (the storage for those numbers), and Chunking (how we split documents into retrievable pieces). This concept shows why retrieval must be permission-aware — that connects to your notes on Admissible Evidence Spans (when only certain passages are allowed) and to Retrieval Quality (because if retrieval fails, the answer is useless). You will also see why this is architecture, not a prompt trick — as your book says, RAG is a pipeline, not a magic prompt.


## The mathematics

This concept has no calculation at its core — the mathematics lies in how we decide whether a document is relevant and whether a user is allowed to see it. Relevance is a similarity score: we measure how close a query is to a document using a geometric idea called a dot product — think of it as measuring 'how much two arrows point in the same direction'. For documents, we turn words into numbers (embeddings) so each document becomes a long list of numbers (a vector). The query also becomes a vector. Then dot product gives a score: high score means the query and document point in the same 'direction' in meaning-space, low score means they don’t. Permission is simpler: we have a list of who can access which documents — a set of rules. We check that before showing anything. There is no arithmetic to do here, but the key mathematical operations are the dot product (for relevance) and set membership (for permission).

Relevance score between query q and document d is the dot product: \( \text{score}(q, d) = q_1 d_1 + q_2 d_2 + \ldots + q_n d_n = \sum_{i=1}^{n} q_i d_i \). Permission is a binary check: \( \text{allowed}(user, doc) \in \{0, 1\} \). Only documents with score > threshold AND allowed = 1 are returned.


## A worked example

```
Imagine each document and each query is a simple 2-number vector (meaning we only have 2 dimensions). Document A: [0.8, 0.2] (meaning it's mostly about 'finance', a bit about 'policy'). Document B: [0.1, 0.9] (mostly about 'HR'). Query: "What is the travel reimbursement policy?" represented as [0.7, 0.3]. We want to see which document is more relevant. Permission: The user has access to Document A but not Document B.

Step 1: Dot product (query, DocA) = (0.7 × 0.8) + (0.3 × 0.2) = 0.56 + 0.06 = 0.62   # Multiply matching numbers and add them up.
Step 2: Dot product (query, DocB) = (0.7 × 0.1) + (0.3 × 0.9) = 0.07 + 0.27 = 0.34   # Same calculation for Document B.
Step 3: 0.62 > 0.34, so Document A is more relevant (higher score)   # Higher dot product means more similar.
Step 4: Permission check: user has access to DocA, not DocB → only DocA is passed to generation   # Even though DocB got a lower relevance, if it had been the only match, we would have to say 'no access' rather than reveal its content.

Result: The retrieval system returns Document A only, because it is both relevant and permitted. The assistant then answers using only that document.
```


## From scratch, in Python

```python
# Let's build a tiny permission-aware retrieval system.
# We have a small list of documents, each with a unique ID, a title, and a simple vector.
# We'll also have a list of users, each with a set of document IDs they can access.

# A list of dictionaries: each dict holds a document's id, title, and vector.
documents = [
    {"doc_id": "doc1", "title": "Travel Reimbursement Policy", "vector": [0.8, 0.2]},
    {"doc_id": "doc2", "title": "HR Leave Policy", "vector": [0.1, 0.9]},
    {"doc_id": "doc3", "title": "Expense Reporting Guide", "vector": [0.7, 0.25]}
]

# Permissions: a dictionary mapping user names to a set of doc_ids they can see.
# A set is like a bag that keeps each item once, and we can quickly check if something is in it.
permissions = {
    "alice": {"doc1", "doc3"},   # Alice can see doc1 and doc3
    "bob": {"doc2", "doc3"}     # Bob can see doc2 and doc3
}

# Our query vector (representing "travel reimbursement policy")
query_vector = [0.7, 0.3]

# A function to calculate the dot product of two vectors (lists of numbers).
# A function is a reusable block of code that takes inputs and returns an output.
def dot_product(vec_a, vec_b):
    result = 0
    for i in range(len(vec_a)):  # loop over each position
        result = result + (vec_a[i] * vec_b[i])  # multiply and add
    return result

# Retrieve relevant documents for a user
# We'll check the dot product and then check permission.
def retrieve(user_name, query, documents, permissions):
    # Get the user's allowed doc IDs
    allowed_docs = permissions[user_name]
    # We'll collect results in a list
    results = []
    for doc in documents:
        if doc["doc_id"] not in allowed_docs:
            continue  # skip documents the user cannot see
        score = dot_product(query, doc["vector"])
        # We'll set a simple threshold: only include if score >= 0.5 (just for this example)
        if score >= 0.5:
            results.append({"doc_id": doc["doc_id"], "title": doc["title"], "score": score})
    # Sort by score descending (highest first)
    results.sort(key=lambda x: x["score"], reverse=True)
    return results

# Test with Alice
print("Alice's results:")
for r in retrieve("alice", query_vector, documents, permissions):
    print(r)

# Test with Bob
print("\nBob's results:")
for r in retrieve("bob", query_vector, documents, permissions):
    print(r)
```


## The same thing, with a framework

```python
In real systems, you'd use a vector database like Chroma or Pinecone, and an embedding model. Here's a sketch using Chroma (pseudo-code):

```
import chromadb
client = chromadb.Client()
collection = client.create_collection("enterprise_docs")
# Add documents with embeddings (you'd generate these from a model)
collection.add(
    documents=["Travel Reimbursement Policy", "HR Leave Policy"],
    metadatas=[{"doc_id": "doc1", "allowed_users": ["alice"]}, {"doc_id": "doc2", "allowed_users": ["bob"]}],
    ids=["doc1", "doc2"]
)
# Query
results = collection.query(
    query_texts=["travel reimbursement policy"],
    n_results=2
)
# Then you filter results based on the user's permissions
# The filter is applied after retrieval, which is a security risk — see the misconceptions.
```
The `dot_product` we wrote by hand is what the database does internally — it computes the distance between query and stored embeddings.
```


## What the code does

**Lists and dictionaries** – `documents` is a list of dictionaries. A list keeps things in order, like a row of lockers. A dictionary looks things up by name rather than by position, like a labelled box: `doc["doc_id"]` gets the key 'doc_id'. **Sets** – `permissions` uses sets (`{...}`) because a set holds each item once and we can quickly check if a document ID is in it with `not in`. **Loops** – `for i in range(len(vec_a))` loops over each index (0, 1, ...) of the vector. **Functions** – `dot_product` and `retrieve` are reusable blocks. `retrieve` calls `dot_product` and checks permission. **Skipping** – `continue` jumps to the next document if the user isn't allowed. **Sorting** – we sort the results by score using a lambda (a short anonymous function). The framework version (Chroma) would do the dot product and search for you, but you still need to enforce permissions — and doing it after retrieval is risky because the database might expose snippets in the search results before you filter.


## Common misconceptions

**Filtering after generation is safe** — I can retrieve everything and then just filter out the documents the user isn't allowed to see before showing the answer.

The model has already seen the forbidden document. It might incorporate its facts into the answer, and you can't reliably detect a leak. In fact: Enforce permission at retrieval time — never let the model even see a document it shouldn't.

**Gaps can be filled with common sense** — If the retrieval doesn't find the exact answer, I can let the model use its general knowledge to fill in the gaps.

That turns the assistant into a general chatbot — it might make things up (hallucinate) or give advice outside the company's policy. In fact: Saying 'I don't have information on that' is a feature — it builds trust.

**Stale documents are not a problem** — Once we've indexed a document, it's always valid. We don't need to revisit them.

Company policies change, and the assistant might give outdated advice. In fact: Retrieval must factor in recency, or the system must flag when a document is old and may be superseded.

**Retrieval is just a search box** — Anything that does keyword search over the documents counts as retrieval.

Semantic relevance matters. A keyword search might miss the exact phrase 'travel policy' if the document says 'reimbursement guidelines'. In fact: Use embedding-based semantic search, possibly combined with keyword search (hybrid) and reranking.


## In short

An enterprise knowledge assistant is only as good as its retrieval. Ground every answer in retrieved, permission-approved passages. Enforce permissions at retrieval, not after. Accept that documents are often stale or contradictory and deal with it honestly. Saying 'I don't know' is a product feature. The architecture is: ingest → chunk → embed → retrieve (with permissions) → generate.


## Where this is used

**In large language models (LLMs):** Enterprise assistants built on LLMs (like GPT-4 or Claude) don't use the model's raw knowledge about your company. Instead, the LLM is given the retrieved documents in its context (the input window) and told 'answer only from these'. This is RAG. If the retrieval returns wrong or unpermitted documents, the answer will be wrong or leak secrets. The mechanism: you query the vector index, get top-k chunks, then place them in the prompt as context. The LLM then generates an answer conditioned on that context. **In small models on devices (TinyML):** On a phone or laptop, you might not have a huge LLM. Instead, you use a smaller model or a retrieval model that runs on-device. The permission checks happen locally. The mechanism: the device has a local index of documents (maybe cached) and uses a lightweight embedding model. Retrieval happens on-device, and generation might be a rule-based summariser or a small language model. The point is that RAG works even without a giant model — the retrieval engine is the core. **In agentic systems that plan and call tools:** An enterprise assistant might be an agent that can query databases, send emails, or update records. The retrieval step is part of the agent's planning. The agent decides to call a retrieval tool, gets documents, then takes actions based on them. The permission check must be embedded in the tool call — the agent cannot bypass it. A typical developer writes a tool like `search_documents(query, user)` that automatically filters by the user's access level before returning. If the agent tries to list all documents, the filter blocks it. **In everyday applied ML (recommendation, fraud detection, etc.):** Any application where you need to ground an AI's output in a specific set of records — e.g., a customer support assistant looking at the user's past orders, or a fraud detection system referencing transaction history. The same retrieval-permission pattern applies: retrieve only the records the user is allowed to see, then feed as context to a model. **What developers actually write:** They build a pipeline: ingest (split into chunks), embed (convert to vectors), store in a vector DB, retrieve (query with user permissions), rerank (optional), then generate. They write code that calls a vector database API, filters results by user roles (using metadata like 'allowed_roles'), and constructs a prompt that says 'answer from these passages only'. They also evaluate retrieval quality separately from generation quality. **What breaks when done badly:** If permission is filtered after generation, the model might have already seen the document and produced an answer that leaks it. Misconception: 'I can filter after the LLM has generated' — wrong, because the LLM might quote the forbidden text. If retrieval returns too many irrelevant chunks, the context is polluted and the answer drifts. If stale documents are not marked, the assistant might cite outdated policy as if current. The assistant must be honest about uncertainty: it should say 'I'm not sure' rather than make up an answer.


---

Read this on the web: https://basavapurushottam.com/learn-ai/enterprise-knowledge-assistants
