Changing the model

Every other technique on this site changes what you send the model, on every call. Adaptation changes the model itself, once, so a later call can be shorter, cheaper or more consistent without repeating the same instructions or examples. Fine-tuning trains further on your own examples. LoRA and other adapters train a small add-on instead of the whole model: the LoRA paper describes freezing the pretrained weights and injecting trainable “rank decomposition matrices” into each layer, and reports that this cuts trainable parameters by 10,000 times and GPU memory by 3 times against fine-tuning GPT-3 175B with Adam[1]. Distillation trains a smaller model on a larger one’s outputs. Reinforcement fine-tuning trains against a scored reward instead of fixed example answers. Synthetic data generates training examples with a model. Automated prompt optimization searches for a better prompt instead of a person hand-editing one.

Adaptation is orthogonal to the eight levels: an adapted model can sit under a single chat call or under one seat in a team of agents. It is the alternative to writing the same instructions into every prompt: prompt engineering’s “say it every time” against adaptation’s “train it in once.”

This page is sourced, not measured: the choice below is described from primary sources, but no training run has happened here, so what follows shows how the data gets prepared and nothing more.

def load_examples(questions_path: Path = DEFAULT_QUESTIONS_PATH) -> list[Example]:
    """Every question in the set, as a training example. Every question carries a plain-language
    `answer` even when its kind is `unanswerable` (the correct completion there is the model
    saying so), so nothing is filtered out by kind."""
    data = json.loads(questions_path.read_text(encoding="utf-8"))
    return [Example(id=q["id"], question=q["question"], answer=q["answer"]) for q in data["questions"]]

def leaked_questions(train: list[Example], val: list[Example]) -> list[str]:
    """Val-set ids whose normalized question text also appears in the train set. A held-out split
    is only worth anything if training never saw the same question under a different id.

    This catches only questions that are identical once normalized. A paraphrase in genuinely
    different words ("how long is the warranty" against "what is the warranty period") is a
    near-duplicate this equality test cannot see; catching those needs a similarity measure, and
    an embedding of each question is the usual one."""
    train_texts = {_normalize(ex.question) for ex in train}
    return sorted(ex.id for ex in val if _normalize(ex.question) in train_texts)

Failure modes

At each level

• Conventional software: a classical model such as the ones on the level 0 page is, by definition, already fit to your data every time it is trained: the questions this page raises about adaptation do not arise until there is a language model to adapt.
• Direct prompting: a fine-tuned model can replace a long, repeated prompt-engineered system prompt with a shorter call that already behaves the trained way, at the cost of retraining whenever the instructions change.
• Added context: adaptation does not substitute for retrieval: training a model further changes how it behaves, not what current facts it can reliably recall, so a fine-tuned model still needs documents in front of it for anything that changes after training.
• Workflows: a fixed step that is called the same way thousands of times, with the same instructions and shape of input every time (one step of prompt chaining, say) is the cheapest place to swap a small adapted model in for a large general one.
• Tool use: a model fine-tuned on your own tool set can produce fewer malformed function calls than a general model prompted with the same tool definitions, since it has seen your schemas specifically rather than schemas in general.
• Agent loops: reinforcement fine-tuning fits a single agent’s loop especially well, because it trains directly against whether the task got done, the same thing the loop’s own stop decision is trying to get right, instead of imitating example transcripts.
• Teams of Agents: a fixed role played by one agent every time (an author, a reviewer) is a narrow, repeated task, the case adaptation is built for; each seat could run its own smaller adapted model instead of every seat running the same large one.
• Always-on agents: an always-on assistant logs every action it proposed and what happened to it, which is exactly the raw material a later fine-tuning or distillation pass would use, and exactly the data the safety, privacy and governance topic’s questions about retention and training use apply to.

Practices

• Before fine-tuning, try the cheaper alternative first: a better prompt, a few-shot example, or retrieval. Adaptation is worth the cost when the same instructions are sent enough times that training once is cheaper than prompting every time, not by default.
• Hold out a validation split and never train on it. A model that has only ever been graded on data it was also trained on has not been graded.
• Check a prepared training file for near-duplicate examples straddling the train/validation split before spending anything on a training run; a leak inflates validation scores without improving anything real.
• Do not treat a fine-tuned model as a substitute for giving it current documents. Ask what the model was trained to do, separately from what it was trained to know.
• Keep the data a fine-tuning job is built from somewhere it can be regenerated or re-audited, the same discipline evals/questions.json follows for this site’s own eval set.
• Four pages under this one go further than this page does on each way of adapting a model: fine-tuning and adapters, distillation, synthetic data and prompt optimization.

Run it

Try it

1. Use it

   Find a product that advertises a "custom-trained" or "fine-tuned" small model. Read what task it claims to be good at, and ask whether that claim is about behavior on that one task or about general knowledge: the page usually only supports the first.

2. Build it

   Run python -m examples.adaptation --out .local/scratch/adaptation from the repo root, open val.jsonl, and check that every line has exactly one system, one user and one assistant message in that order.

3. Either lane

   Pick two questions from evals/questions.json that ask about the same appliance in different words, and decide whether they're similar enough that putting one in training and one in validation would leak. Write down what made the call.