MLOps: Getting Models Into Production and Keeping Them There

A model that works in a notebook is an experiment. A model that works in production, keeps working as the data changes, can be rolled back when it doesn't, and has someone accountable for it — that's a product. MLOps is the set of practices that gets you from the first to the second. It borrows from DevOps but adds the things software engineering never had to deal with: data as a dependency, models that decay, and correctness that's statistical rather than exact.

Why ML is harder to operate than software

Conventional software has one moving part: code. An ML system has three — code, data and the model — and a change in any of them changes behaviour. The data pipeline that feeds training can silently shift. The model's performance degrades over time even if nothing is deployed, because the world moves on. And "correct" is a distribution of outcomes, not a pass/fail test. Most ML projects that fail after launch fail on these operational fronts, not on modelling.

The pipeline

The unit of work in MLOps is a reproducible pipeline, not a script. A typical one:

1. Data ingestion and validation — pull from sources, check schema, ranges, null rates and distribution against expectations. Fail loudly when the data looks wrong.
2. Feature engineering — ideally through a feature store that computes features once and serves them identically for training and inference, eliminating the training/serving skew that causes so many silent bugs.
3. Training — with every input (data snapshot, code commit, hyperparameters, environment) recorded so the run can be reproduced.
4. Evaluation — against a held-out set and against the current production model, with thresholds that gate promotion. Include slices (by region, segment, demographic) so a regression in a subgroup can't hide in the average.
5. Registration — the candidate model, its metrics and its lineage are stored in a model registry with a version and a stage (staging, production, archived).
6. Deployment — packaged and released through the same CI/CD discipline as code.

Orchestrators (Airflow, Kubeflow Pipelines, Prefect, Dagster, or the cloud providers' managed equivalents) run this; experiment trackers and registries (MLflow, Weights & Biases, SageMaker, Vertex AI) record it.

Deploying safely

Never swap a model in place. The options, in increasing safety:

• Shadow deployment: the new model receives production traffic and its predictions are logged but not used. You compare against the incumbent on real data at zero risk.
• Canary release: a small percentage of traffic goes to the new model; expand if metrics hold.
• A/B testing: a controlled split with business-metric measurement, when the goal is to know whether the model actually improves outcomes rather than just accuracy.
• Blue-green: two full environments; switch traffic atomically and switch back if needed.

Every deployment needs a rollback that takes minutes. Model registries make this a version pointer change.

Monitoring: the part that's different

Standard observability — latency, throughput, errors, cost — applies. On top of it, ML needs:

• Input drift: are the features arriving in production distributed like the training data? Statistical tests per feature, alerting on divergence.
• Prediction drift: has the distribution of outputs shifted? A fraud model that suddenly flags 3× more transactions is telling you something.
• Performance when ground truth arrives — which may be immediately (click or no click) or months later (loan default). Design the feedback loop deliberately.
• Data quality upstream: a schema change in a source system is the most common cause of a model quietly breaking.

Alerts should route to whoever owns the model, and a drift alert should trigger a defined action: investigate, retrain, or roll back.

LLMOps: the same discipline, new problems

Systems built on large language models need everything above plus some new pieces. Prompts and retrieval configurations are code and must be versioned and tested. Evaluation can't be exact-match; it needs curated test sets, rubric-based scoring (often by another model), and human review of samples. Tracing across multi-step agent workflows is essential for debugging. Cost and latency per request vary wildly with prompt length and model choice. And guardrails — input filtering, output validation, prompt-injection defences — are runtime components that need their own monitoring. The tooling (LangSmith, Langfuse, Arize, Braintrust and the platform vendors' offerings) has matured quickly, but the practice is still settling.

Ownership

The most common MLOps failure is organisational: data scientists build, someone else deploys, nobody owns what happens next. The teams that succeed give a model a named owner responsible for its production behaviour, staff a platform team that provides the pipeline and monitoring infrastructure so every project doesn't rebuild it, and treat a model in production as a service with an SLO, an on-call rota and a runbook — because that's what it is. Governance requirements — documentation, audit trails, approval gates — slot naturally into a pipeline that already records everything.