Large language models in business software
We build large language models such as Anthropic's Claude into existing business systems through their APIs, with answers drawn from your own records, tested before release and kept within your data protection rules.
What you probably want to know first
- Is our data used to train the model?
- Not under the business API terms of the major providers we use, which do not train on customer data by default. We check the terms and data region for each provider before using it, and send the model only what each task needs.
- How do you stop it making things up?
- The model is given the relevant records and documents for each question and asked to answer from them, with links to its sources. Answers are tested against a set of real questions before release, and anything that writes to your system goes through a person or a rules check first.
- What will it cost to run?
- Model usage is charged per request, and for most business uses it is small next to the staff time saved. We estimate it from your volumes before building, choose the smallest model that does the job well, and set limits so the cost cannot run away.
- What have you built with language models?
- We use Claude in our own engineering: every code change on the OpenIRIS project gets an automated review, and test plans for each sprint are drafted from the changes in it. We have also built search over a library of guidance documents that answers from those documents and cites them.
Who this page is for
This page is for IT managers who have been asked whether AI could help with a business system, and want to know what building it in actually involves before committing budget.
What a language model is good for in business software
Large language models are good at reading unstructured text and producing text: pulling fields out of a document, answering a question from a set of records, drafting a reply, sorting incoming messages. They are not reliable on their own as a source of facts, calculations or decisions. The useful designs give the model the right information for each request, ask it for something specific, and check what comes back.
How we build it
The model is called from your application’s server, never from the browser, so keys and data stay under control. Search over your own records decides what the model sees for each request. The output is checked: by validation rules where it becomes data, by a person where it goes to a customer. A test set of real examples runs before every release, because a change to a prompt or a model version can alter results in ways nobody would notice by eye.
We use the same approach in our own work, where Claude reviews every code change on OpenIRIS and drafts the test plan for each sprint, with people deciding what to act on.
Talk to us
Describe the system and what you need. You will hear back from someone who can answer technical questions.
Discuss AI for your system 0800 433 7990What the technical work involves
- Model APIs
- Claude and other models called from your application's server, with keys held securely and usage logged.
- Answers from your own data
- Relevant records and documents found by search and given to the model with each question, so answers are grounded and cited.
- Structured output
- Documents read into fields your system can store and validate, not free text.
- Testing answers
- A set of real questions with known good answers, run before each release to catch regressions.
- Cost and speed control
- The right model for each task, caching where it helps, and limits on spend.
- A person in control
- Anything that changes records or goes to a customer is reviewed or checked against rules first.
How we add a language model to a system
Pick one task
A single, measurable piece of manual work, such as reading one type of document.
Collect real examples
Past documents or questions with the correct result, used to build and test the feature.
Build behind a switch
The feature runs alongside the current process, and its results are compared with what staff did.
Go live with review
Results are checked by a person until the error rate is known and acceptable.
A good fit when
- Staff spend hours re-keying documents into a system.
- People struggle to find answers that are already in your documents and records.
- You want to try AI on one well-defined task before committing further.
Probably not for you if
- You want a general chatbot with no connection to your systems or data.
- The underlying system is unstable; that is worth fixing first.
Questions we are asked
Do you only work with Claude?
No. We choose the model for the task, its cost and the provider's data terms. Claude is the one we use most.
Can the model run on our own servers?
Some open models can, at the cost of lower quality on harder tasks and the work of running them. We compare that with a hosted model for your case.
Tell us about your system
Say what it does, what it is built on and what is worrying you. We will reply with what we would look at first and whether we are the right people to help.