Back-office document AI needs reliability, grounding and workflow control

A Singapore engineering and facilities services contractor asked us to automate part of its finance workflow. The team checks supplier invoices against purchase orders and delivery orders before payment. Some documents arrive by email. Most arrive as hard copy, then get scanned on a copier or photographed on site. We tested SmartDoc against direct frontier model calls on their documents. On the decision that matters, approve the payment or flag it, SmartDoc agreed with the human reviewer on all 24 discrepancy checks. Claude Sonnet 5 and Gemini 3.5 Flash each approved document sets that contained real errors. SmartDoc also read 97.4 percent of fields correctly across more than 150 scored document runs, against 91.2 for Claude Sonnet 5 and 89.6 for Gemini 3.5 Flash.

This is the pattern behind many back-office processes. The business needs a system that can turn messy documents into reliable records, check those records against business rules, show evidence for each answer and route exceptions without forcing staff to inspect every line.

A specialized document processing platform adds four things that direct model calls do not reliably provide on their own:

1. It returns higher field accuracy on messy operational documents, where copier scans, phone photographs and carbon-copy delivery orders carry the real workload.
2. It returns the same answer when the same document is processed again, which matters when the output gates a payment decision.
3. It grounds each extracted value to the page, so a reviewer can inspect the source of a field instead of trusting a free-text answer.
4. It breaks work into inspectable stages: parse, extract, compare and route. That makes errors diagnosable and gives downstream systems structured data they can use.

The difference matters because every later step depends on the fields produced at the start. Matching, approval routing, exception handling and ERP posting all inherit the extraction layer's errors. If the field is wrong, the workflow can still produce a clean-looking result.

Use three-way invoice checking as the test case

Before finance pays a supplier, someone confirms that the invoice matches the purchase order and delivery order. The purchase order records what the company agreed to buy. The delivery order records what arrived on site. The invoice records what the supplier wants paid. The check catches wrong quantities, wrong prices, undelivered items and items billed under the wrong description.

Today the work is manual. Finance blocks out an hour to scan and upload, then reviews the next day. The check gates payment, so the team cannot read every line at higher volume.

The documents below show the shape of the use case.

The invoice quantity for line 2 matches the delivery order, but the invoice description says 1000 mm while the purchase order and delivery order say 1500 mm. The arithmetic is right. The record is wrong.

The document AI checker needs to answer specific questions:

1. Do all three documents refer to the same PO number, supplier and delivery site?
2. Does each invoice line map to a purchase order line and delivery order line?
3. Did the supplier invoice only the quantities that were delivered?
4. Did the invoice preserve the ordered item description, item code and unit price?
5. Does the line amount equal quantity multiplied by unit price, and does the subtotal reconcile?
6. Which exact page region supports each extracted field, so finance can review exceptions quickly?

Page reading is only the first step in workflow control

Frontier models can read scanned invoices and flag mismatches with purchase orders. A meeting-room demonstration makes the task look simple because the model answers once, on one document, with a person watching.

The production job has a different contract. It means thousands of documents a month, limited human attention and a payment decision at the end. The system must give repeatable outputs, show the source of each field and degrade safely when the input document is poor.

SmartDoc is built around that contract. The benchmark tested whether the extra structure mattered against direct frontier model calls using the same documents, prompts and field schemas.

SmartDoc led on accuracy, verdicts, stability and grounding

The customer gave us multiple document sets. Each set contained a purchase order, a delivery order and an invoice, including real scans and phone photographs. We tested three engines: SmartDoc, Gemini 3.5 Flash and Claude Sonnet 5. The test produced more than 150 scored document runs and multiple comparison checks per engine. The two frontier models used the same prompts and field schemas as SmartDoc.

The findings were clear:

1. SmartDoc led on field accuracy: 97.4 percent against 91.2 percent for Claude Sonnet 5 and 89.6 percent for Gemini 3.5 Flash. That gap becomes operating load because every wrong field becomes an exception, a manual recheck or a silent error downstream.
2. SmartDoc produced the strongest payment-control result. It called every scored discrepancy check correctly with no false passes. The frontier models missed real invoice errors, which matters because a false pass lets a bad document set move through the gate.
3. SmartDoc was more stable on reruns. The same document produced the same parse most of the time, while the frontier models changed their answers across repeated runs. A back-office control cannot change its answer when the source document did not change.
4. SmartDoc returned page grounding for extracted values. Direct model calls returned answers without source boxes, so the reviewer had to verify claims by rereading the document.

The findings mean the platform is doing more than calling a model. It gives the workflow a reliable data layer, a repeatable control point and an audit trail that finance can use when a payment is held, released or escalated.

A reliable control gives the same answer on a re-run

The accuracy gap was clear. The consistency gap was larger.

Given the same page three times, SmartDoc returned byte-identical parses on most repeated comparisons. Both frontier models returned none. Field-value stability across the three runs was 0.977 for SmartDoc, 0.904 for one model and 0.863 for the other.

One engine read the same line-item block as 70 items, then 5, then 70. On another set, a discrepancy it caught on two runs came back as a clean pass on the third.

A check that changes its answer when the document did not change is not a reliable check.

Small accuracy gaps become wrong fields and wrong payment calls

Each three-document set averaged 56 extracted fields. At 1,000 sets a month, 56,000 fields pass through the process.

Seven points of field accuracy becomes about 4,300 more wrong fields a month, or 52,000 a year. Each wrong field becomes an exception someone works or an error nobody catches.

The verdicts pointed the same way. SmartDoc called 24 of 24 discrepancy checks correctly. Gemini missed one. Claude missed three. Each miss passed a set that contained a real error. At 2,000 checks a month, those miss rates mean Gemini would approve roughly 83 document sets a month that contain real errors, and Claude roughly 250, without anyone knowing.

One case made the risk concrete. A purchase order listed two lengths of the same part. The invoice printed the cheaper description twice. The totals still reconciled to the cent, so the money was right and the record was wrong. That kind of drift surfaces later in a dispute.

Staged processing makes errors traceable

A direct model call can take both documents and return a verdict. SmartDoc runs the job in three stages, with each stage creating an artefact that can be inspected, scored and reused.

Parse converts the page into structured text. Extract reads that text against a schema and returns typed fields. Compare checks those fields against a reference document.

Each stage hands the next a named artefact, so the team can see whether an error came from reading the page, extracting the field or comparing the documents. Grounding attaches at the extraction layer, which gives the customer structured fields it can write into its own systems whether or not it runs the comparison.

This is the practical difference between frontier model capability and production document processing. The model supplies the reasoning and language capability. The platform supplies the operating contract: schema, grounding, repeatability, staged artefacts, exception handling and integration into the systems that finance already uses.

Benchmark note: these findings come from one customer test set and should be read as directional. Results vary with document quality, layout diversity, workflow rules and the customer's own tolerance for exceptions. Cost was measured differently across engines, so this article does not compare cost.