Topics at every level
Five topics cut across all the levels. Each has its own set of pages.
Every level
Five topics that apply at every level
Evaluation frameworks
SourcedThe tools that run test sets and graders for you, and what to check before trusting their numbers.
Fine-tuning and adapters
SourcedTraining a model further on your own examples, in full or with small adapters such as LoRA.
Synthetic data
SourcedUsing a model to write training or test examples, and checking them before they are used.
Guardrails
SourcedChecks on what goes into a model and what comes out, and the limits of those checks.
Red teaming
SourcedAttacking your own system on purpose, before someone else does, and turning what you find into tests.
Observability
SourcedRecording what each run did, so a bad result can be traced to the step that caused it.
AI gateways
SourcedOne entry point in front of several model providers, for keys, routing, limits, fallback and logs.
Cost optimization
SourcedSpending fewer tokens and less time for the same result: caching, batching, smaller models, shorter context.
Running models locally
SourcedRunning open-weight models on your own hardware: what fits, quantization, and what you give up.
Briefing: saying what you want
SourcedSaying what you want clearly enough that the model does not have to guess.
Deciding what to hand over
SourcedDeciding which parts of a task to hand to a model and which to keep.
Calibrating trust
SourcedLearning, from results over time, how much to rely on a model without checking.
Named products, tools and models
Names listed 09/19/2026. 223 of 223 registry entries have been checked against the maker's own page; the registry marks the rest as unchecked.
- AI Guardrails (Lakera Guard)Check Point · prompt-injection filter · formerly Lakera Guard
- Axolotlopen source · fine-tuning library
- BraintrustBraintrust · eval platform
- DeepEvalConfident AI · eval framework
- distilabelArgilla · synthetic data
- DSPyStanford NLP · prompt programs and optimizers
- garakNVIDIA · LLM vulnerability scanner
- Guardrails AIGuardrails AI · guardrails framework
- HeliconeHelicone · tracing and cost tracking
- InspectUK AI Security Institute · eval framework
- LangfuseClickHouse · tracing and cost tracking
- LangSmithLangChain · eval and tracing platform
- LiteLLMBerriAI · one API for many models
- Llama Guard 4Meta · safety classifier · formerly Llama Guard
- llama.cppopen source · runs models locally
- LM StudioElement Labs · runs models locally
- lm-evaluation-harnessEleutherAI · benchmark runner
- MLXApple · training and inference on Apple hardware
- NeMo GuardrailsNVIDIA · guardrails framework
- OllamaOllama · runs models locally
- OpenAI EvalsOpenAI · eval frameworkRetires 2026-11-30
- OpenAI fine-tuningOpenAI · hosted fine-tuningRetired 2026
- OpenRouterOpenRouter · one API for many models
- OpenTelemetryopen standard · tracing standard
- PEFTHugging Face · LoRA and other adapters
- PhoenixArize AI · AI observability and evaluation
- promptfoopromptfoo · eval runner
- Ragasopen source · evals for retrieval
- SGLangSGLang community · model serving runtime
- Together AI fine-tuningTogether AI · hosted fine-tuning
- TransformersHugging Face · model library
- TRLHugging Face · fine-tuning library
- UnslothUnsloth · fine-tuning library
- vLLMopen source · model server
Pages at this level last reviewed 09/19/2026. Pages unreviewed for 90 days are flagged for another pass. Markdown version of this page