# Multi-Tenant Adapter Serving

Serving many customers from one base model by loading their individual adapters on demand.

**Rests on:** Foundation Models, Pre-Trained Model Adaptation

*LLMs and Generative AI · Fine-Tuning*


## Why learn this

This concept bridges the gap between training a custom AI model and actually using it in a real application. If you have trained a specialized version of a language model for your company or a specific client, you now face a practical problem: how to serve that model to many users simultaneously without spending a fortune on hardware. Multi-tenant adapter serving allows you to pay for the computing power of just one base model while simultaneously serving hundreds of customized versions of it. Mastering this gives you three things: the ability to optimize how models run, the skill to evaluate them fairly, and the practical knowledge needed to build a platform that hosts these models. It also reveals why methods like parameter-efficient fine-tuning exist—this is the payoff for using techniques like LoRA (Low-Rank Adaptation) and QLoRA (Quantized Low-Rank Adaptation). Without this approach, your work remains stuck on a single laptop and never reaches a real user.


## The intuition

Imagine you run a small restaurant with one popular dish: a base curry recipe. Your regular customers all order it, but each person tweaks the seasoning: Ravi wants extra chili, Priya wants no onions, and Mrs. Shah wants extra garlic. You do not cook each bowl from scratch, which would require reheating the entire large pot every time—that is wasteful and slow. Instead, you keep one big pot of base curry simmering on the stove all day long. When a customer orders, you take their portion of the base curry and quickly stir in their specific extras from small jars you have prepared beforehand. 

The big pot of base curry represents the 'base model'—the general AI intelligence kept active in memory. The individual add-ons—the chili, the missing onions, the extra garlic—are the 'adapters.' You serve each customer by combining the base with their specific adapter at the exact moment they order. This means you use one kitchen and one stove, but every plate is customized. In the AI world, 'multi-tenant adapter serving' means keeping one large language model loaded in memory continuously, but for every new question or request, loading only the small adapter that belongs to that specific user and running the computation through it. The heavy lifting is done by the base model just once; the customization is light and fast. The economic benefit is clear: you do not need a separate, expensive graphics card (GPU) for each customer’s customized model. You simply need one powerful computer running the base, plus small adapter files that slot in and out instantly per request.


## An analogy

Think of a large hotel kitchen that serves diverse menus. The hotel has one central commercial kitchen (the base model) equipped with standard appliances. To handle specific dietary needs or cuisines without rebuilding the kitchen every time, they hire specialized chefs who can be called in temporarily. One hour, a guest orders traditional Japanese kaiseki; the chef for that style arrives. The next hour, a vegetarian Italian requests a risotto; a different specialist chef arrives. The main kitchen stays exactly the same—the stoves and ovens do not move. Only the small team of specialists changes to guide the preparation for that specific meal. This works because the specialists are few, small in number, and quick to start working. They do not replace the kitchen itself; they merely direct it. 

Where this analogy stops being perfect is in speed: in a real hotel, calling a chef might take minutes of phone calls and walking across the lobby. In AI adapter serving, swapping an adapter must be nearly instantaneous—taking only milliseconds—because user requests arrive one after another without pause. Also, in a physical kitchen, two chefs might accidentally bump into each other or leave ingredients that contaminate the next dish. In software, the system must strictly isolate every tenant so that one user's data or instructions never leak into another user's service.


## Definition

Multi-tenant adapter serving is the method of keeping a single shared base model in active memory and, for each incoming request, quickly loading a small, task-specific adapter file associated with the requesting user to produce customized outputs, ensuring the swap happens fast enough for real-time use and maintaining strict isolation so users never interfere with one another.


## How it connects

In your studies, this sits directly under Fine-Tuning. It relies on parameter-efficient fine-tuning techniques—methods that allow you to modify a model using only a tiny fraction of its original parameters—and specifically uses the adapters created by Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA). These techniques are what make the adapter files small enough to swap quickly. It also relates to Inference Optimization, which is the practice of making AI models run faster and cheaper when responding to users, and Post-Training Alignment, which is the process of teaching a model to follow instructions safely. This approach also solves the problem of catastrophic forgetting—the phenomenon where a model loses its previously learned general skills when forced to learn new, specific tasks—by keeping the main base model frozen in place. The adapters carry only the new knowledge, leaving the original training untouched.


## The mathematics

This concept draws on arithmetic first, and a little bit of linear algebra — but we will start with ordinary numbers. You know that a model takes inputs and produces outputs. Think of the base model as a machine that takes in text and produces an answer, using its internal arithmetic: lots of additions and multiplications. When we fine-tune with adapters, we're not changing that machine; we're adding a small extra set of numbers (the adapter's weights) that sits alongside the base. To produce an answer with a tenant's adapter, the model does something like this: it computes the base answer, but it also computes an extra adjustment from the adapter, and it adds the two together. The adapter's contribution is a small correction, often visualised as a delta — a difference. The combined result is base plus delta. In numbers, if the base would produce a score of 5 for a certain token, and the adapter shifts it by +0.3, the final score is 5.3. The model uses these adjusted scores to decide what to output. That's the entire mathematics of adapter inference: a weighted sum of base and adapter outputs, with the adapter's weights being small. The key arithmetic is: you have a base matrix (a big table of numbers) and an adapter matrix (a small table), and the operation is addition of their products — but we'll do a tiny example with real numbers to see it.

For each token that the model predicts, the final score with an adapter is:

output = base_score + adapter_score

where base_score combines the base model's weights with the input, and adapter_score combines the adapter's weights with the same input. In the language of matrices, if the base model produces a vector of scores b, and the adapter produces a smaller vector a (which is added to the relevant part of b), the final vector is b + a. Each symbol: b is the base model's raw output scores, a is the adapter's contribution, and the plus means you add them element by element (first numbers together, second numbers together, and so on).


## A worked example

```
Suppose we have a base model that gives a score to a word. For the sentence 'The cat sat on the mat', the base model gives a score of +2.5 to the word 'cat'. A tenant adapter, trained to make the model talk about animals, adds a +1.2 to that score. Another tenant, trained for legal text, adds a -0.8 instead. We'll calculate the final scores for both tenants.

Step 1: Base score for 'cat' = 2.5   # The base model alone, without any adapter, assigns this score.
Step 2: Tenant A (animal) adapter adds +1.2   # The adapter is a small set of numbers that, when applied, adjusts the base score.
Step 3: Final score for Tenant A = 2.5 + 1.2 = 3.7   # By adding the adapter's adjustment to the base, we get the tenant-specific score.
Step 4: Tenant B (legal) adapter adds -0.8   # A different adapter, a different adjustment.
Step 5: Final score for Tenant B = 2.5 + (-0.8) = 1.7   # The same base score, but the adapter shifts it differently.

Result: The same base model, with different adapters, produces different scores for the same word. In practice, the model uses these scores to choose the next word in the answer. The adapter's contribution is a small addition, but it can change the entire meaning of the response.
```


## From scratch, in Python

We'll simulate a tiny base model and two adapters. We'll have a function that, given an adapter, adjusts the base's output.

```python
# We don't have a real model, so we'll simulate the base model's scoring
# with a simple dictionary: for each word, the base model gives a 'score'.
base_scores = {
    'cat': 2.5,
    'dog': 1.8,
    'contract': 0.2,
    'court': 0.1
}

# Adapters are just dictionaries that say how much to adjust each word.
adapter_animal = {'cat': 1.2, 'dog': 0.9}  # boosts animal words
adapter_legal = {'contract': 0.8, 'court': 1.1}  # boosts legal words

# A function that takes an adapter and returns adjusted scores.
def serve_with_adapter(adapter):
    # Start with the base scores
    final_scores = {}
    for word, score in base_scores.items():
        # Start with the base score
        final = score
        # Add the adapter's adjustment if it has one for this word
        if word in adapter:
            final = final + adapter[word]
        final_scores[word] = final
    return final_scores

# Serve two requests: one for tenant A (animal), one for tenant B (legal)
print("Tenant A (animal):", serve_with_adapter(adapter_animal))
print("Tenant B (legal):", serve_with_adapter(adapter_legal))
```


## The same thing, with a framework

In practice, you'd use a framework like PyTorch or Hugging Face's `peft`. Here's how it looks with `peft`:

```python
from transformers import AutoModelForCausalLM
from peft import PeftModel

# Load the same base model once (pseudo-code, you'd specify a real model)
base_model = AutoModelForCausalLM.from_pretrained("gpt2")

# Each tenant has their own adapter file
adapter_a = PeftModel.from_pretrained(base_model, "./adapters/tenant_a")
adapter_b = PeftModel.from_pretrained(base_model, "./adapters/tenant_b")

# Simulate a request: you load the adapter, run the model, then unload
# In reality, you'd swap adapters in and out per request, but here's the concept:
def serve_request(user_id, prompt):
    if user_id == "tenant_a":
        model = adapter_a
    else:
        model = adapter_b
    output = model.generate(prompt)
    return output
```


## What the code does

In the from-scratch code, we use a Python dictionary — a data structure that looks up a value by a key (like a word) rather than by a position. The `base_scores` dictionary simulates the base model's output for a few words. The `adapter_animal` and `adapter_legal` are dictionaries mapping a word to the amount by which the adapter changes the base's score. The function `serve_with_adapter` loops over each word in `base_scores`, checks if the adapter has a different adjustment for that word (using `word in adapter`), and if so adds it to the base score. This simulates the per-request adapter swap: you pass in the adapter for the tenant, and the function produces a customised set of scores. In the framework version, we use Hugging Face's `transformers` and `peft`. We load the same base model once, and then load each tenant's adapter as a `PeftModel` attached to that base. In a real serving setup, you'd load the adapter on demand for each request, run the model, and then remove it to free memory. The code shows the key idea: one base model, many small adapters, and you select which adapter to use based on the user.


## Common misconceptions

**A separate model per tenant** — To serve each client, I need to load a full copy of the model for each one.

That would cost as much as buying a GPU per client. The whole point of adapters is to avoid that. In fact: You load one base model and swap small adapters. The base is the expensive part; the adapter is cheap. You only need as many simultaneous copies of the base as you have concurrent requests, not as many tenants.

**Adapter swapping is slow and causes downtime** — Every time a customer changes, the model has to reload, which takes minutes and blocks other users.

Modern serving systems pre-load adapters or swap them in milliseconds by keeping the base resident and applying the adapter's weights on the fly. In fact: Well-engineered systems keep the base model constantly loaded and only swap the small adapter's weight matrices, which is fast enough to happen per request without noticeable delay.

**Tenant isolation is just about preventing interference** — As long as one tenant's adapter doesn't accidentally change another's output, we're fine.

Isolation is also a security boundary. A malicious tenant might try to craft an adapter that leaks another tenant's data or prompts, and the system must prevent that. In fact: Tenant isolation means strict isolation of memory, state, and inference. A serving system must ensure that one tenant cannot read or modify another's adapter weights or influence their outputs beyond the intended model behaviour.

**Adapters can be loaded randomly for any request** — I can just keep a dictionary of adapters and pick any one at any time.

Memory and latency constraints mean you can't keep all adapters loaded simultaneously. You must evict and load them based on demand, using caching strategies. In fact: A serving system manages a pool of adapters, keeping the most-used ones in memory and loading others on demand, with eviction policies based on recency or frequency.


## In short

Multi-tenant adapter serving is the deployment-side payoff of PEFT: keep one base model in memory, and for each incoming request, load the tenant's small adapter — the base and adapter combine to produce a customised output. The swap must be fast and secure. This is how one GPU serves hundreds of customised models. Remember: the base is the expensive shared resource; the adapter is the cheap, isolated, per-tenant variation. If you remember one thing: one base, many adapters, loaded per request.


## Where this is used

Where is multi-tenant adapter serving used in practice? This is the backbone of many AI-as-a-service products. Let's go through each area:

- **Large language models (LLMs)**: OpenAI, Google, and others offer fine-tuning APIs where a customer uploads data, and the provider returns a fine-tuned model. But behind the scenes, they don't spin up a separate GPU per customer. They run one base model (like GPT-4 or Gemini) and store each customer's fine-tuned adapter (often LoRA) in a database. When a customer sends a request, the serving platform loads that customer's adapter, applies it to the base, generates the response, and then discards the adapter. This keeps costs low: the base model is the most expensive part, and the adapter is tiny (a few megabytes vs. hundreds of gigabytes). The developer using such an API simply calls the model endpoint with an `adapter_id` or similar; the platform handles the swap transparently. If this is done badly, request latency spikes because swapping adapters takes time, or worse, adapters leak into each other's requests — a security disaster.


---

Read this on the web: https://basavapurushottam.com/learn-ai/multi-tenant-adapter-serving
