AI and machine learning that survives contact with production

Where AI pays off, and where it does not

The useful question is never “how do we use AI”. It is “which decision in this business gets made badly, often, and at scale”. The first question produces an assistant nobody asked for; the second produces something that quietly removes a repetitive step from a team’s day, and it is rarely the thing that photographs well in a board pack.

Machine learning works best where three conditions hold at once. There is a pattern that is real but too intricate to write down as rules. There are enough recorded examples of it to learn from. And being wrong occasionally is tolerable, because no useful model is right every time. Remove any one of those and the honest recommendation changes: a rules engine, a better report, or nothing at all.

Google’s own Rules of Machine Learning opens by telling engineers not to be afraid to launch a product without machine learning, on the grounds that where a model might deliver a large gain, a heuristic will get you roughly half of it straight away. That is a sequencing argument rather than a discouraging one: ship the heuristic, learn what the data looks like in production, and let the measured gap tell you whether the model is worth building.

The questions we ask before writing any code

What decision changes? If the answer is “we would know more”, the project is analytics rather than machine learning, and it belongs with the reporting work.

Which direction of wrong is worse? Missing a fraudulent transaction and blocking a legitimate one are not symmetrical, and the threshold that balances them is a business decision made either deliberately or by accident.

Where would the answer appear? A prediction that arrives in a weekly export nobody opens is worth nothing; one that appears in the screen where the decision is already being made is worth a great deal, which makes this as much an interface design problem as a modelling one.

Who is accountable for the output? Somebody has to own the model being wrong. If no name goes in that box, the system should not go live.

Data readiness is the part nobody gets to skip

The gap between a promising idea and a working model is almost always data, and almost never algorithms. The algorithms are largely commodity: scikit-learn, PyTorch and a handful of well-documented architectures cover most of what a mid-market business needs. Your data is the part nobody else has, and it is also the part that turns out to be messier than the enthusiasm suggested.

What we look at, in order. Coverage: does the data actually contain the outcome you want to predict, recorded when it happened rather than reconstructed afterwards. Labels: are the examples labelled, by whom, and consistently. Balance: how rare is the event, because rarity rather than volume is usually the binding constraint. Leakage: does a field in the training data encode the answer in a way that will not exist at prediction time. Leakage is the most common reason a model scores brilliantly in development and collapses in production, and it hides in innocuous places — a status field updated after the fact, a timestamp that only exists for resolved cases.

Joins and definitions: whether “customer” means the same thing in the CRM and the billing system. This is the least glamorous item on the list and the one that most often decides the timeline. Where the pipelines and definitions need work first, that is data and analytics engineering, and we would rather do it in the right order than train a model on a table nobody trusts.

The output of this phase is a written readiness report, including the sentence nobody enjoys writing: this dataset cannot answer that question, and here is what would have to exist before it could.

Classical machine learning wins more often than it gets credit for

Large language models have absorbed the entire conversation, which has quietly made a lot of organisations worse at the problems they actually have. If the task is to predict which customers will lapse next quarter, which invoices will pay late, how much stock a branch needs in March, or which of tonight’s transactions deserve a human look, a gradient-boosted tree on tabular data will usually beat a language model, cost a fraction as much to run, and produce feature importances a domain expert can argue with.

The discipline here is well understood: careful feature engineering, honest cross-validation, a decision threshold chosen against the real cost of each error type, and calibration so that a predicted probability means what it says. Time-series problems need splits that respect chronology, because shuffling rows lets a model learn from the future and produces a validation score that is pure fiction.

We use PyTorch and TensorFlow where the problem genuinely calls for deep learning — unstructured inputs, images, audio, sequences — and scikit-learn or XGBoost where it does not, which is more often than the market currently pretends. Where the model has to run somewhere awkward, exporting to ONNX lets it be trained in Python and served inside a C#, C++ or Java application, or on a device, without rewriting the maths.

Language models: assistants, extraction and the boring wins

The highest-value language-model work we see is rarely the chatbot. It is extraction, classification and summarisation applied to text previously handled by a person copying between systems: pulling structured fields out of supplier emails, routing inbound enquiries, drafting a first response for a human to approve, reducing a long case file to the four facts the next person needs.

These are good problems for language models because they are bounded, the output can be checked, and the alternative is expensive human attention on work nobody enjoys. They also fail safely: a wrong extraction is corrected by the person who was going to read the document anyway.

Prompt engineering is a real engineering activity here, not a folk art. The prompt lives in version control, its structure stays stable enough for caching to work, its instructions are specific about output format, and every change is scored against the same evaluation set rather than judged by whoever tried three examples. The major providers now expose broadly comparable primitives — the OpenAI API, the Anthropic Claude API and the Google Gemini API all accept structured output constraints and tool definitions — so a well-built integration keeps the vendor behind an interface of your own.

Retrieval before fine-tuning

The most common request we get is some version of “make it know our stuff”. Almost always that is retrieval rather than fine-tuning: retrieval keeps knowledge in a store you can update this afternoon, while fine-tuning bakes it into weights that will confidently repeat last year’s policy until somebody retrains them. Fine-tuning is the right tool for form rather than fact.

Retrieval-augmented generation on your own content

A retrieval-augmented generation system has three parts, and only one of them is the model. An ingestion pipeline turns your documents into chunks with metadata. A retrieval step finds the chunks relevant to a question, usually by embedding similarity, often improved considerably by combining that with ordinary keyword search. A generation step answers from those chunks and cites them.

The infrastructure choice matters less than teams expect. Postgres with pgvector handles a great many corpora perfectly well and keeps your content in a database you already back up and already secure; a dedicated vector database such as Pinecone earns its place at larger scale. Reaching for the exotic option first is a common and expensive mistake.

Where retrieval systems actually fail

Not usually in the model. In the plumbing.

Chunking destroys context. A table split across two chunks answers nothing. Chunk on document structure rather than a fixed character count, and keep the heading trail attached to the text.

Permissions are not enforced at retrieval time. If the index does not filter by who is asking, the assistant becomes an exfiltration route for every document it has ever read. This sits in the OWASP GenAI project’s catalogue of vector and embedding weaknesses, and it is the failure most likely to end a project after launch rather than before it.

The index goes stale. Documents change; nobody built the refresh. Six months later the assistant is quietly authoritative about a superseded process.

Retrieval is never measured separately. Teams evaluate the generated answer and never check whether the right passage was retrieved at all, so when the answer is wrong they tune the prompt — which cannot fix a retrieval miss. Measure the two stages independently or you will spend months optimising the wrong one.

Nothing is cited. An answer with no link to its source has to be trusted rather than checked, which is precisely the property you do not want.

Agents, and the honest version of workflow automation

An agent is a language model given tools and permission to decide which to call. Used carefully — a small set of well-described tools, a bounded task, a hard limit on steps, and confirmation before anything irreversible — it is a genuinely useful way to automate a multi-step process that resists flowcharting.

Used carelessly it is a liability, and the OWASP GenAI Top 10 names the failure modes precisely: excessive agency, where a system can take actions well beyond what the task requires, and unbounded consumption, where a loop runs until the invoice arrives. Prompt injection makes both worse, because content the agent reads can contain instructions the agent then follows.

Our position is that agents get read access broadly and write access narrowly. Anything that spends money, contacts a customer or deletes data goes through a person or a hard-coded rule rather than a model’s judgement. Every tool call is logged with its inputs, outputs and the path that led there, because when the thing does something strange at three in the morning, that log is the alternative to a rebuild. And there is always a step budget, because “it will stop when it is done” is not a control.

Computer vision and document processing

Vision work divides cleanly into problems that are solved and problems that are research. Classification, object detection, reading text from documents, inspection against a reference, counting things in a frame — these are engineering, using OpenCV for the pipeline, established architectures for the model, and ONNX Runtime where inference has to happen on a device rather than in Python. Understanding an unconstrained scene, or inferring intent from behaviour, is research, and we will say so rather than quote for it.

Document processing is where most of the commercial value sits, because it is where the manual work sits: invoices, delivery notes, forms, contracts. The pattern that works is layered — a managed extraction service such as Amazon Textract or Google Document AI for general structure, a specialised model for the fields your business actually cares about, and a confidence threshold that routes anything doubtful to a person. The aim is not to eliminate human review, but to reduce it from every document to the ones the system is genuinely unsure about, and to make that boundary explicit rather than accidental.

Evaluation: proving it is good enough to ship

Evaluation is the difference between a demo and a system, and it is the part buyers are most often asked to accept on faith.

It begins with a held-out set. The scikit-learn documentation states the principle without decoration: learning the parameters of a prediction function and testing it on the same data is a methodological mistake, because a model that merely repeats what it has already seen scores perfectly and predicts nothing. So a portion of the data is set aside at the start and stays out, and k-fold cross-validation is used within the training data when a single split would waste too much of it.

Then the metric has to match the decision. Accuracy is close to meaningless on imbalanced problems — a model that predicts “not fraud” for every transaction scores extremely well and is worthless. Precision, recall and the trade-off between them are what a business actually feels, and the threshold that sets that balance should be chosen with the person who owns the consequences in the room.

Generative features need a different apparatus, because there is rarely one correct string: a scored rubric applied per item, a fixed set of representative cases including the awkward ones, automated checks for what is checkable — output format, groundedness in the retrieved passages, refusal to answer when the corpus does not support an answer — and human review of a sample. Model-graded evaluation is useful, and it is not a substitute for a person reading real outputs.

Then a shadow run against live traffic, recorded and compared but never shown to a customer. It remains the cheapest way to find out that production data does not resemble your training data.

Guardrails, human review, and being wrong well

Every model is wrong sometimes. The design question is what happens when it is, and answering that well is most of the difference between a system that survives its first bad week and one that gets switched off.

Confidence thresholds route uncertain cases to people instead of guessing. Output validation checks structure and plausibility before anything downstream consumes the result. Rate and spend limits cap the damage from a loop. Refusal behaviour is defined, so an assistant with no supporting document says it does not know rather than inventing something — misinformation is on the OWASP list for good reason, and a fluent wrong answer is more dangerous than an obviously broken one. Prompt injection defences assume that any content the system reads may contain instructions, and that tool permissions, not clever wording in the prompt, are what contain the blast radius.

Human review is designed rather than bolted on: deciding which decisions require a person, giving them enough evidence to judge quickly, and capturing their corrections as training data so the loop closes. A review step that shows a bare verdict with no context does not produce review; it produces rubber-stamping.

Running it: deployment, drift and cost control

A model that lives in a notebook is a research artefact. Production means versioned artefacts, reproducible training runs, a deployment path, monitoring, and a way back.

We track experiments, parameters, metrics and artefacts in MLflow, so that six months later somebody can answer which data and which code produced the model currently making decisions. Training and serving run in containers, orchestrated with Kubernetes where the scale justifies it. Where inference belongs inside the application rather than behind a network call — a web application doing real-time scoring, or a mobile app that has to work offline — an ONNX export moves the model to where the decision happens.

Monitoring covers three things that are often conflated. Operational health: latency, errors, throughput. Input drift: whether the data arriving now still resembles what the model learned from. Outcome quality: whether predictions are still right, which is hardest to observe because the truth often arrives weeks later, if at all. Where ground truth is delayed, proxy signals — override rates, correction rates, complaint volume — are what you actually watch, and they have to be instrumented deliberately.

Drift is not a defect. It is the normal condition of a model in a world that keeps changing, so the plan is agreed up front: what triggers a retrain, who approves it, how the new version is compared against the old on the same held-out set, and how you roll back within the hour if it turns out worse.

What actually drives the running cost

For a classical model the cost is mostly infrastructure and retraining, and it is fairly predictable. For anything built on a hosted language model it is usage-based, which means it can surprise you.

The dominant term is almost always input tokens, and the dominant part of those is the context you resend on every call rather than anything the user typed. That makes the levers concrete. Prompt caching lets a stable prefix be served from cache instead of reprocessed, but it matches on an exact prefix, so a timestamp near the top of a prompt silently disables the saving for everything after it. Better retrieval cuts input directly: three well-chosen passages beat thirty mediocre ones on both cost and answer quality. Routine steps can run on a smaller model while hard steps escalate. Work that does not need an immediate answer can go through asynchronous batch processing rather than the real-time path.

None of that is manageable without measurement, so we log token counts and cost per request from the first deployment. A team that cannot attribute its bill to a feature cannot reduce it.

Governance, privacy and the regulatory picture

Regulation is now a design input rather than a compliance afterthought. In the EU, the AI Act sorts systems into tiers — a small set of prohibited practices, a high-risk category carrying substantial obligations around risk management, data governance, logging and human oversight, a transparency tier requiring that people know they are dealing with AI and that generated content be identifiable, and a minimal-risk remainder. Duties differ depending on whether you are the provider of a system or its deployer, and the application calendar has already been amended once, which is reason enough to check the Commission’s own page rather than a summary written a year ago.

The NIST AI Risk Management Framework is voluntary and worth following anyway: its four functions — Govern, Map, Measure, Manage — produce almost exactly the artefacts a regulator, an enterprise procurement team or an insurer will ask for, and building them alongside the system costs a fraction of assembling them afterwards under time pressure.

Then there is the law you already live under. Data protection rules apply to personal data whether or not a model is involved: lawful basis, minimisation, retention, and the position of an individual when a decision is made about them automatically. Our contribution is the documentation and the plumbing — what data goes where, under which agreement, retained how long, with which human check on the way out — and a clear statement of where our competence ends and your counsel’s begins.

How this connects to the rest of our work

Machine learning is rarely the first thing a business needs. More often the sequence runs the other way: the reporting has to be trustworthy before a model can be trained on it, which makes data and analytics the prerequisite rather than the follow-up. Once a model exists it has to appear somewhere useful, which usually means the website or platform where the decision is already being made. And where the output is uncertain — a probability, a ranked list, a generated draft — presenting it honestly is a design problem in its own right, because the alternative is misplaced trust or a feature everyone learns to ignore.

If you are still weighing up whether a use case is worth building at all, that is a conversation we are happy to have before any engagement exists. Tell us which decision you are trying to improve, and we will say whether a model is the right instrument for it.