Foundation Models: Why One Model Now Does Many Jobs

Until recently, every AI application needed its own model, trained from scratch on its own labelled data. A spam filter, a translator and an image tagger shared nothing. Foundation models broke that pattern: train one very large model on a broad slice of the world's text, code and images, and it becomes a starting point for almost any task with little or no additional training. This is the single biggest change in how AI is built, and it explains why the field moved so fast after 2020.

What makes a model "foundation"

The term, coined by Stanford researchers in 2021, describes models with three properties: trained on broad data at scale, usually by self-supervision (predicting the next token, filling in masked regions); general-purpose, capturing knowledge and skills not tied to one task; and adaptable to many downstream uses through fine-tuning, prompting or tool integration. Large language models are the best-known examples, but the category includes vision models, speech models, protein and molecule models, and increasingly multimodal models that handle several of these at once.

How they're built

Pretraining is the expensive step: a transformer with billions to over a trillion parameters is trained on trillions of tokens to predict the next one. The cost runs to tens or hundreds of millions of dollars in compute for a frontier model, which is why only a handful of organisations train them from scratch. The striking empirical finding of the past few years — scaling laws — is that capability improves predictably with more data, more parameters and more compute, and that new abilities appear as scale increases.

Post-training turns a raw next-token predictor into something useful. Supervised fine-tuning on curated examples teaches format and instruction-following; reinforcement learning from human or AI feedback shapes helpfulness and safety; and, since 2024, reinforcement learning on verifiable tasks (maths, code) has produced reasoning models that spend extra compute "thinking" before answering. Much of the recent progress has come from this stage rather than from bigger pretraining runs.

Adapting a foundation model

You rarely train a foundation model; you adapt one. The options, roughly in order of cost:

• Prompting. Describe the task, give examples, set constraints. With modern instruction-tuned models this covers a surprising share of use cases with zero training.
• Retrieval-augmented generation (RAG). Fetch relevant documents from your own data at query time and put them in the prompt. This is how models answer questions about information they were never trained on, with citations, and it's the dominant enterprise pattern.
• Tool use and agents. Give the model functions it can call — search, databases, code execution, APIs — and let it decide when. This extends a static model into a system that can act.
• Parameter-efficient fine-tuning (LoRA and relatives). Train a small number of additional weights on your data while freezing the base. Cheap, and it captures style, domain vocabulary and formats well.
• Full fine-tuning or continued pretraining. Expensive; justified for specialised domains (law, medicine, a proprietary codebase) where the base model's knowledge is genuinely insufficient.
• Distillation of a large model into a small one you can run cheaply or on-device — see Edge AI.

A common mistake is reaching for fine-tuning when the problem is missing information (which RAG solves) or unclear instructions (which prompting solves).

Open weights vs closed APIs

Frontier models from OpenAI, Anthropic and Google are accessed through APIs; you get the best capability and none of the operational burden, at the cost of data leaving your environment and dependence on a vendor's roadmap. Open-weight models — Meta's Llama, Mistral, Alibaba's Qwen, DeepSeek, Google's Gemma and others — can be downloaded and run anywhere, fine-tuned freely and audited. The capability gap between the two has narrowed dramatically, and most organisations end up using both: closed models for the hardest tasks, open ones where control, cost or privacy dominate.

Risks that come bundled in

• Hallucination. Models generate plausible text, not verified text. Grounding via RAG and requiring citations reduce it; nothing eliminates it.
• Bias and toxicity absorbed from training data, mitigated but not removed by post-training.
• Prompt injection. When a model reads untrusted content (a web page, an email) and can take actions, that content can hijack it. This is the central security problem of agentic systems and has no complete solution yet.
• Data leakage and memorisation of training data, and of your data if it's used for further training under permissive terms.
• Concentration and cost. A few providers, enormous compute requirements, and — for open models — the question of who audits what went into them.

The EU AI Act now imposes documentation, copyright and safety obligations on general-purpose model providers, and evaluation frameworks from NIST, the UK AI Security Institute and others are becoming the reference for what "tested" means. Using a foundation model responsibly means treating it as a powerful, imperfect component — evaluated on your task, monitored in production, and never the last line of defence.