# Pre-Trained Model Adaptation

The general practice of taking an existing trained model and adjusting it for a new task or domain rather than training from scratch.

**Rests on:** Foundation Models

**Leads to:** Catastrophic Forgetting, Classification Fine-Tuning, Continued Pre-Training, Federated Learning, FinBERT and BloombergGPT, Instruction Tuning, Knowledge Distillation, Local Model Deployment, Multi-Tenant Adapter Serving, On-Policy vs Off-Policy, PEFT, Supervised Fine-Tuning, Synthetic Data Generation

*LLMs and Generative AI · Fine-Tuning*


## Why learn this

You already know that a large language model is a neural network with billions of weights, trained on vast amounts of text. If you wanted to turn that general-purpose tool into a financial assistant, a medical chatbot, or a customer-support agent, you have two choices: build it entirely from scratch, or tweak what already exists. Building from scratch takes immense time and computational power—resources most organizations do not have. Tweaking it allows a single base model to specialize for specific jobs without retraining the whole thing from zero. This process lets you take a broadly skilled model and give it a narrow, specialized polish through targeted practice on your own data.


## The intuition

Imagine you have a brilliant general-purpose translator who speaks ten languages fluently but has never heard of stock-market jargon. When you need them to translate a financial report, you don't send them back to school for ten years to relearn every language from the alphabet up. Instead, you sit them down with a week of financial articles, letting them pick up the specific vocabulary and phrasing used in that industry. That week of targeted practice is adaptation. The translator already knows grammar, reasoning, and general knowledge from their original training. Adaptation simply nudges them toward this new domain so they don't have to relearn everything from zero. It is about adding a layer of specialized expertise to an already strong foundation.


## An analogy

Think of a chef who trained at a famous culinary school. They can cook French, Italian, and Japanese cuisine brilliantly, but they've never touched Indian street food. To add that to their repertoire, you wouldn't send them back to culinary school for three years. You would give them two weeks in a Mumbai kitchen, learning the regional spices and techniques. That two-week immersion is adaptation; the chef already knows how to handle a knife, manage heat, and plate dishes—they just need the new domain's specifics. Where the analogy stops: a chef's training changes their entire approach to cooking, but adapting a model often leaves most of its core knowledge untouched. Instead of changing the chef's hands, it is more like giving them a new spice rack or an extra station at the counter—small, targeted additions that allow them to produce specific results without rewriting their entire culinary instinct.


## Definition

Pre-trained model adaptation is the practice of taking an existing model that has already been trained on a large, general dataset and adjusting it—either by updating some of its internal weights or by training additional small modules—to perform better on a new, narrower task or domain, without starting the training process from the beginning.


## How it connects

This concept builds directly on your understanding that a large language model is a neural network with billions of weights. Instead of training a new network from random weights, you start from the existing one and continue training on a smaller, focused dataset. This connects to the technique known as transfer learning, which is the broader idea of carrying knowledge gained from one task over to help learn another. It also relates to fine-tuning, which refers specifically to the process of taking that pre-trained model and adjusting its parameters to improve performance on a specific downstream application.


## The mathematics

The core mathematical concept in fine-tuning is not a new type of calculation, but a rule about which numbers get changed and how they are changed.

A model is made of weights. A weight is just a number that scales an input. When you train a model from scratch, you start with random numbers for every weight. You then adjust all of them to fit the data. When you adapt (fine-tune) a pre-trained model, you start with the numbers the model already has. These numbers are good at one task. You want to keep most of them fixed because they still hold useful knowledge. You only change a small part of them to help with the new task.

There are two main ways to do this mathematically:

1. Freezing weights: You decide that some weights will never change. Their value stays exactly the same during the entire adaptation process. Only the other weights, called 'trainable weights', are updated.

2. Adding adapters (like LoRA): Instead of changing the big existing weights, you add small new matrices alongside them. The original large weight matrix W is frozen. You introduce two smaller matrices, A and B. The model now computes using the sum of the original weight and the product of the new ones: W + BA. Because A and B are small, there are far fewer numbers to update. This saves memory and computation.

The update rule for any trainable weight is based on gradient descent. Gradient descent is a method to find the bottom of a valley. Imagine the error of the model as a landscape with hills and valleys. You want to reach the lowest point (minimum error). The 'gradient' tells you which way is uphill. To go downhill, you move in the opposite direction.

The formula for updating one trainable weight w_i is:

w_i_new = w_i - learning_rate * gradient_i

Here is what each part means:
w_i is the current value of the weight.
w_i_new is the new value after this step.
learning_rate is a small positive number, like 0.1 or 0.001. It controls how big a step you take. If it is too big, you might jump over the bottom of the valley. If it is too small, it takes forever to get there.
gradient_i is the slope of the error at the current weight value. It tells you how much the error changes if you change w_i slightly. Mathematically, it is the derivative of the loss function with respect to w_i.

If a weight is frozen, its gradient is ignored, and its value does not change:
w_frozen_new = w_frozen

In LoRA, the effective weight becomes W + BA. During training, you update A and B. After training, you can combine them into one matrix: W_final = W + BA. This new matrix is then used for prediction, just like any other weight.

The key mathematical insight is that by freezing most parameters or using low-rank updates (where R is much smaller than C and D), you reduce the number of variables in the optimization problem. This makes it easier to find a good solution with less data and less computing power.


## A worked example

Let us look at a tiny model with two weights, w1 and w2.

The model computes: output = w1 * x1 + w2 * x2

Suppose the current weights are:
w1 = 0.5
w2 = -1.0

We have one training example for the new task:
x1 = 2, x2 = 3
desired_output = 1.0

Step 1: Compute the current output.
output_current = 0.5 * 2 + (-1.0) * 3
output_current = 1.0 - 3.0
output_current = -2.0

Step 2: Compute the error.
We use squared error as the loss function.
error = (output_current - desired_output)^2
error = (-2.0 - 1.0)^2
error = (-3.0)^2
error = 9.0

Step 3: Determine the gradient for each weight.
The gradient is how much the error changes when we change a weight.
For w1:
gradient_w1 = 2 * (output_current - desired_output) * x1
gradient_w1 = 2 * (-2.0 - 1.0) * 2
gradient_w1 = 2 * (-3.0) * 2
gradient_w1 = -12.0

For w2:
gradient_w2 = 2 * (output_current - desired_output) * x2
gradient_w2 = 2 * (-2.0 - 1.0) * 3
gradient_w2 = 2 * (-3.0) * 3
gradient_w2 = -18.0

Note: The gradient is negative because increasing the weight would make the output more negative (further from 1.0 if we consider the sign, but actually here we want to increase output from -2.0 to 1.0. Let's re-evaluate the direction.
Wait, if output is -2.0 and target is 1.0, we need to INCREASE the output.
The derivative of (y_hat - y)^2 with respect to w is 2*(y_hat - y)*x.
Here y_hat = -2, y = 1. So (y_hat - y) = -3.
x1 = 2. So gradient_w1 = 2 * (-3) * 2 = -12.
The update rule is w_new = w - lr * grad.
w1_new = 0.5 - lr * (-12) = 0.5 + 12*lr.
So increasing lr increases w1, which increases output. This is correct.

Step 4: Update the weights with a learning rate of 0.1.
Let lr = 0.1.
w1_new = 0.5 - 0.1 * (-12)
w1_new = 0.5 + 1.2
w1_new = 1.7

w2_new = -1.0 - 0.1 * (-18)
w2_new = -1.0 + 1.8
w2_new = 0.8

Step 5: Compute the new output.
output_new = 1.7 * 2 + 0.8 * 3
output_new = 3.4 + 2.4
output_new = 5.8

Wait, the output went from -2.0 to 5.8. The target is 1.0. It overshot.
This is because the learning rate was too large for this single step with these gradients.
Let's try a smaller learning rate, lr = 0.01.
w1_new = 0.5 - 0.01 * (-12) = 0.5 + 0.12 = 0.62
w2_new = -1.0 - 0.01 * (-18) = -1.0 + 0.18 = -0.82

output_new = 0.62 * 2 + (-0.82) * 3
output_new = 1.24 - 2.46
output_new = -1.22

Step 6: Compute the new error.
error_new = (-1.22 - 1.0)^2
error_new = (-2.22)^2
error_new = 4.9284

The error dropped from 9.0 to 4.9284. The adaptation is working.

Step 7: Compare with freezing one weight.
Suppose we freeze w2 (w2_frozen = -1.0). We only update w1.
gradient_w1 is still -12.
w1_new = 0.5 - 0.01 * (-12) = 0.62
output_new = 0.62 * 2 + (-1.0) * 3 = 1.24 - 3.0 = -1.76
error_new = (-1.76 - 1.0)^2 = (-2.76)^2 = 7.6176

The error dropped from 9.0 to 7.6176. This is a smaller improvement than updating both weights (4.9284). This shows that freezing weights limits the model's ability to minimize error on the new task, which is why we only freeze some or use adapters.

Step 8: LoRA example.
Suppose w1 is part of a large matrix W. We replace W with W + BA.
Let W = [0.5]. Let A be a 1x1 matrix [a] and B be a 1x1 matrix [b].
R=1, C=1, D=1. R << min(C,D) is not strictly true here for simplicity, but let's say we use low rank.
Initially, B is initialized to 0. So BA = 0.
Effective weight = W + BA = 0.5 + 0 = 0.5.
We train a and b.
Let a = 0.1, b = 0.2.
BA = 0.1 * 0.2 = 0.02.
Effective weight = 0.5 + 0.02 = 0.52.
The original W (0.5) is frozen. Only a and b are updated.
This requires storing only a and b, not the full gradient for W.


## From scratch, in Python

```python
# We'll simulate a tiny model with two weights, and adapt them on a small dataset.
# This demonstrates the core idea: starting from existing weights, adjusting them with small steps.

weights = [0.5, -1.0]  # a list of two numbers, our starting point
training_data = [
    ([2, 3], 1.0),   # input [2,3] should output 1.0
    ([1, -1], -1.0), # input [1,-1] should output -1.0
]
learning_rate = 0.1

# A simple function to compute the model's output for a given input
def predict(inputs):
    # dot product: multiply each input by its weight, sum them
    total = 0
    for i in range(len(inputs)):
        total = total + inputs[i] * weights[i]
    return total

# Training loop: repeat many times
for epoch in range(100):
    # for each example, compute error and nudge weights
    for inputs, target in training_data:
        output = predict(inputs)
        error = output - target
        # update each weight (here, we update all weights; freezing would skip some)
        for i in range(len(weights)):
            # the gradient for weight i is roughly error * inputs[i]
            gradient = error * inputs[i]
            weights[i] = weights[i] - learning_rate * gradient

print(weights)
print(predict([2,3]))  # should be close to 1.0
```


## The same thing, with a framework

In PyTorch, you would create a model, freeze some layers, and train the rest. Here is a minimal example using a small neural network:

```python
import torch
import torch.nn as nn
import torch.optim as optim

# A tiny model with one linear layer
model = nn.Linear(2, 1)

# Freeze all weights (we'll unfreeze the bias later, but let's see freezing)
for param in model.parameters():
    param.requires_grad = False

# Unfreeze the bias so it can adapt
model.bias.requires_grad = True

# Optimizer only sees the trainable parameters
optimizer = optim.SGD(filter(lambda p: p.requires_grad, model.parameters()), lr=0.1)

# Suppose we have a dataset of input-target pairs
data = [(torch.tensor([2.,3.]), torch.tensor([1.])), (torch.tensor([1.,-1.]), torch.tensor([-1.]))]

for epoch in range(100):
    for x, y in data:
        optimizer.zero_grad()
        output = model(x)
        loss = (output - y).pow(2).mean()
        loss.backward()
        optimizer.step()

print(model.weight, model.bias)
```

Under the hood, `loss.backward()` computes the gradients (the same as our manual error * input), and `optimizer.step()` updates the weights using the learning rate, just like our loop did.


## What the code does

The from-scratch code starts with a list of two weights. A list in Python holds values in order — here, `[0.5, -1.0]`. We define a `predict` function that does the dot product: multiply each input by the corresponding weight and sum them. The training loop runs 100 epochs. In each epoch, we go through each example. We compute the output, find the error (difference between output and target), then for each weight we compute a gradient — roughly error times the input — and nudge the weight in the opposite direction of the gradient, scaled by the learning rate. This is exactly gradient descent. We update all weights, but in real adaptation you might freeze some — meaning you skip them in the update. The framework version in PyTorch does the same thing, but with a library call: you tell the model which parameters need gradients, and the optimizer handles the updates. The key line is `param.requires_grad = False` to freeze, and `True` to unfreeze.


## Common misconceptions

**Adaptation means training the whole model again** — I have to retrain the entire network from scratch to adapt it.

Adaptation starts from the existing weights and only adjusts a small subset, or adds new modules. The bulk of the weights stay frozen. In fact: You only update a tiny fraction of the parameters — often less than 1% — while keeping the base model's weights untouched.

**A higher learning rate is always better for faster adaptation** — I'll set the learning rate high so the model adapts faster.

A high learning rate can overshoot the minimum and cause the model to diverge, losing what it already knew. In fact: Adaptation usually uses a lower learning rate than training from scratch, to make small, safe nudges.

**Freezing layers means the model can't learn anything new** — If I freeze most of the model, it won't be able to learn my new task.

Frozen layers retain general features (like grammar and reasoning), and only the last few layers (or adapters) are trained to specialise. This often works better because it prevents forgetting. In fact: Freezing the lower layers preserves general knowledge; only the top layers adapt to the new task.

**Adaptation and prompting are the same thing** — I can just put a long prompt describing the task and it's the same as adapting.

Prompting doesn't change the model's weights; it only changes the input. The model's behaviour may improve, but it's temporary and limited by the context window. In fact: Adaptation permanently modifies the model's weights, so the behaviour change persists even without a detailed prompt.


## In short

Pre-trained model adaptation is taking an existing trained model and giving it targeted training on a new task or domain. You don't start from scratch — you start from the pretrained weights, decide which layers to freeze and which to train (or whether to add small adapter modules), and use a low learning rate to nudge the model. The key decisions are the domain gap size (how different the new data is) and the risk of catastrophic forgetting. This is the foundation of all fine-tuning techniques you will learn next.


## Where this is used

In large language models, adaptation is everywhere. When OpenAI or Google fine-tunes a base model to follow instructions, they use instruction tuning — feeding it thousands of instruction-response pairs and adjusting its weights. When a company like a bank wants a model that understands financial language, they use continued pre-training or domain-adaptive fine-tuning, feeding it years of financial reports. In small models running on devices, like a smartphone's voice assistant, the model is often too big to retrain; instead, developers use parameter-efficient fine-tuning like LoRA — they train small adapter matrices that plug into the frozen base, keeping the model's core knowledge intact while adding a lightweight layer for the specific task, like wake-word detection. In retrieval-augmented generation, adaptation isn't about the model's weights at all — it's about the retriever: you adapt the embedding model (which maps text to vectors) to your domain so that the retrieved documents are more relevant. This is often done by fine-tuning the embedding model on domain-specific pairs. In agentic systems that plan and call tools, adaptation often means fine-tuning the model to know when to call which tool — giving it examples of correct tool use and updating its weights so it plans better. In everyday applied machine learning, adaptation appears in transfer learning: a vision model trained on ImageNet is fine-tuned to detect defects in factory parts, or a sentiment model is adapted to financial news. Developers write code using libraries like Hugging Face Transformers; they load a pretrained model, freeze all but the last few layers, and train on a few thousand labelled examples. What breaks when done badly: if you adapt too aggressively (high learning rate, too many epochs), the model forgets its general knowledge — catastrophic forgetting. If you don't adapt enough, the model doesn't learn the domain. If you use the wrong chat format when fine-tuning a conversational model, the model produces gibberish even though the training loss is low.


---

Read this on the web: https://basavapurushottam.com/learn-ai/pre-trained-model-adaptation
