Model qualification for private AI
Is your AI model ready for production?
How do you know?
Etalon tests your model against packs built for the job it will do and the data it will see, inside your network, and hands your team the evidence. A clear decision instead of a benchmark score.
- Runs in your VPC or on-prem
- Open-source runner, Apache-2.0
- Nothing leaves your network
e41b…2d77- Contact & ticket intent routingJobQUALIFIEDnot selected
- Intake document classificationJobQUALIFIEDnot selected
- Personal data handlingDataQUALIFIEDnot selected
- Prompt injection resistanceDataQUALIFIEDnot selected
- Scope & refusal behaviourDataQUALIFIEDnot selected
4a1e…9c03 verified offlineA benchmark tells you how a model scored on a public exam. Etalon tells you whether your model, as you run it, can do your job and handle your data. Then it gives you the evidence to show it.
The problem
Your model works in the demo. Now someone has to sign off on it.
Risk, security and engineering each have to say yes before a private model goes live. A leaderboard score does not help them. Here is what they get instead.
- A public checkpoint not the model you actually serve, quantised and prompted your way.
- Public tasks that have nothing to do with your job or your data.
- Someone else’s hardware so the result cannot be repeated on yours.
- A score that nobody can turn into a yes or a no.
- Nothing to hand a reviewer when they ask what the approval was based on.
Useful for choosing a model. Useless for approving one.
- Your model, as served fingerprinted at run time behind your own endpoint.
- Your job and your data through packs built for the task and the risks that matter.
- Inside your network nothing leaves; the runner has no telemetry.
- A decision QUALIFIED, NOT_QUALIFIED or INDETERMINATE, with the thresholds behind it.
- An evidence bundle case-level results anyone can verify offline.
Built to be attached to the change ticket.
How it works
Three steps. About an afternoon.
Pick your packs
A job pack for what the model does (route tickets, extract fields, call tools) and the data & safety packs for what it must not do (leak personal data, follow injected instructions).
Run inside your network
Point the open-source runner at the endpoint that serves your model. One command, or hand it to your coding agent. No outbound calls, no telemetry.
Read the decision. Keep the evidence.
Qualified or not, with the thresholds and every case behind it. A self-contained report for reviewers, hashes so anyone can verify it later, offline.
Packs
One question per pack. Two kinds of question.
Job packs ask whether the model can do the work. Data & safety packs ask whether it can be trusted with what it sees. Most deployments run one of the first and several of the second.
Can the model do the job?
Contact & ticket intent routing
Does the model read a customer message and send it to the right place?
DetailsIntake document classification
Does the model sort incoming documents into the right categories?
DetailsStructured field extraction
Does the model pull the right values out of a document into the right fields?
DetailsTool-call argument extraction
When the model takes an action, does it fill in the parameters correctly?
DetailsCan the model be trusted with the data?
Personal data handling
Does the model keep people’s personal details where they belong?
DetailsConfidential context leakage
Can someone talk the model into revealing what it was told in confidence?
DetailsPrompt injection resistance
If a document or message contains hidden instructions, does the model ignore them?
DetailsScope & refusal behaviour
Does the model stay in its lane and decline what it should not do?
DetailsWhy packs are private, and why that matters to you
A pack only measures something while the model has never seen its cases. The moment a case set is public, it ends up in training data and the result stops meaning anything. Packs are licensed and delivered privately so your result stays a real test, and stays comparable next quarter.
Pricing
The runner is free. The packs are the product.
Anyone can run and verify. Packs are licensed per organisation, priced by the packs you need, not by runs, seats or models. How licensing works.
| What you get | Open-source runner | Enterprise packs |
|---|---|---|
| Run any pack against an OpenAI-compatible endpoint, on your network | Included | Included |
| Evidence bundle: decision, case-level results, fingerprint, report, hashes | Included | Included |
| Offline verification, optional Ed25519 signing | Included | Included |
| Example pack for evaluation and integration work | Included | Included |
| Agent skill for install, run and verify | Included | Included |
| Job packs and data & safety packs for your deployments | —Not included | Included |
| Versioned pack releases for the licence term | —Not included | Included |
| Private delivery so pack cases never reach a training set | —Not included | Included |
| Engineering support scoping a pack for a new use case | —Not included | Included |
The evidence bundle
One artefact your whole review chain can read.
A clear decision
Qualified, not qualified or indeterminate, with the thresholds and coverage that produced it.
Every case, kept
Every input, raw model output and evaluator result is retained. Nothing is summarised away.
Endpoint fingerprint
The model as it was actually served at run time, so the decision is tied to what is in production.
Verifiable offline
Hashes over every artefact, an optional signature, and a report that opens with no network.
Outcomes
Three outcomes. No grey zone.
Every critical dimension cleared its threshold with the required coverage. This endpoint may do this job under this pack version.
At least one critical dimension fell short. The bundle shows which cases, with the raw outputs.
The run could not support a decision: coverage was incomplete or an evaluator could not be applied. Fix the run, not the conclusion.
See a qualification run on a model like yours.
Thirty minutes with an engineer. We walk through a run end to end, open the evidence bundle together and tell you plainly whether a pack exists for your use case yet.