Act 1 · The manual process
Every consulting engagement is governed by a contract that defines scope, deliverables, methodology, and acceptance criteria. Verifying that work product conforms to those terms is currently manual, expensive, and error-prone. This demo identifies that manual process, presents an automated solution, and measures the impact.
Section B
A senior manager at any professional services firm performs the following sequence today, by hand, for every engagement that requires a compliance review.
Receives executed contract from engagement team.
Typically 20–50 pages of negotiated scope, deliverables, methodology, acceptance criteria, exclusions, and special terms.
Reads through the contract, identifying clauses relevant to the work product under review.
Notes specific methodology requirements, scope inclusions and exclusions, and deliverable specifications.
Receives draft work product from delivery team.
Typically a technical report, analysis, or recommendation document.
Begins systematic cross-check.
Reads each section of the work product against the relevant contract clauses, looking for scope drift, methodology divergence, missing deliverables, or undisclosed substitutions.
Compiles findings into a compliance memo.
Each finding cites the contract clause it relates to and the work-product item it concerns.
Reviews findings with engagement partner; drafts revisions request to the delivery team.
The end of one review; the beginning of the next.
Total time per review: 4–8 hours of senior manager time, depending on contract complexity and work-product size.
A firm doing professional services work may run hundreds of these reviews per year. Aggregate cost runs into the hundreds of thousands of dollars annually, with material risk exposure from missed compliance issues.
Section C
Manual
Total: 4–8 hours.
Automated
Total: 30–45 minutes, mostly in human review of flagged items.
Time per review: 4–8 hours → 30–45 minutes.
Section D
The system uses two architectural patterns worth understanding before watching it run.
Retrieval-augmented validation. When the system reviews a work product, it does not ask a language model “is this correct?” in the abstract. Instead, for every substantive item in the work product, it retrieves the most relevant passages from the governing contract and asks the model to evaluate the item against those specific passages. This is a deliberate inversion of how most people have encountered AI tools, where models generate content from prompts. Here, the model judges existing content against retrieved specifications. The shift matters because compliance is, by definition, a question of conformance to specification — and grounding judgment in the specification itself produces traceable, auditable verdicts rather than opinions.
Multi-step verification. No single check, deterministic or AI-based, is sufficient on its own. Schema checks catch structural problems but miss semantic ones. Arithmetic checks catch reconciliation failures but miss methodology divergences. Language-model judgment catches semantic issues but can be misled by ambiguous text. The system therefore layers six verification stages, each catching a different category of error, with deterministic checks running first and AI-based judgment running on the structurally-validated remainder. An issue must pass every layer to make it to the final output. The architecture's trustworthiness comes not from any individual stage but from the discipline of layering them.
The pipeline that runs next is an instance of these two patterns applied to contract compliance review. The patterns themselves generalize.
Three sample engagements; pick one to walk through the pipeline.
Bridge to Act 2
Each sample is a real-shaped model-validation services engagement — contract plus draft validation report. The system runs the same six-stage pipeline against all three. The findings differ.
Illustrative engagements · client names use Microsoft sample-data placeholders (Northwind, Contoso, Fabrikam) and are not real institutions.
Replacing approximately 4–8 hours of senior manager review. Estimated time for this run: ~30 seconds.
0.0s
Engagement
Act 2 deliverable
Bridge to Act 3
The dashboard below shows what production deployment of the automation looks like across a synthetic corpus of recent compliance reviews. Coverage and edge-case routing are first-class concerns: the system is trustworthy when it knows what to defer, not only when it knows what to decide.
Mean RAG-validate verdict confidence per ISO week.
Mean findings produced per review per ISO week.
Each row is a past review whose output was routed for additional human judgment.
Act 3 · The measured impact
The architecture above is interesting on its own terms. Whether it is useful is a separate question, and one that requires real numbers. The sections below quantify the change against the manual process described in Act 1.
| Manual | Automated | |
|---|---|---|
| Time per review | 4–8 hours | 30–45 minutes |
| Cost per review (loaded) | $1,000–2,000 | $100–180 |
| Reviewer fatigue at high volume | Significant | None |
| Audit trail | Variable, narrative | Comprehensive, structured |
| Catches subtle methodology divergences | Inconsistent | Systematic |
Sample 3 — the methodology divergence case.
Manual track
Automated track
Same finding, ~960× faster, with a complete audit trail.
Categories of finding the automation reliably surfaces and that manual review tends to miss.
The Sample 3 case study. Subtle replacements of contracted analytical methods with alternatives that look superficially compliant.
Manual review tends to miss this because it's the kind of error that only surfaces if the reviewer reads the methodology section of the contract immediately before reading the methodology section of the work product, every time, fully attentive — which the third such review of the day rarely receives.
Minor inclusions of out-of-scope analysis or recommendations not authorized by the contract.
Manual review tends to miss this because reviewers are looking for what's missing, not what's quietly extra.
Contracted deliverables silently omitted from the work product.
Manual review tends to miss this when the contract's deliverable list isn't actively in front of the reviewer during the read.
Claims in the work product that don't trace to source data.
Manual review tends to miss this on numerical claims that look plausible on first read.
Manual cost is one number multiplied by another: senior manager hours per review, times the loaded hourly rate. There is no API or compute component — the work is done by hand.
Automated cost has two components. The first is a per-review fee for the LLM and embedding API calls plus the compute the pipeline runs on. The second is a block of senior reviewer time — the same reviewer who would have done the entire manual cross-check, now adjudicating only the items the system flagged. The pipeline does not eliminate senior judgment; it compresses the senior reviewer's involvement from the full review to the flagged items. That compression is where the savings live.
Manual cost per review
No API or compute component.
Automated cost per review
Savings per review
Senior hours recovered / yr
Annual savings
Each pipeline run does roughly:
Subtotal: $0.39. The constant used here ($1.50) is rounded up generously to absorb retries, larger contracts, and the quality-judge running on the full work product text. Holds for typical model-validation engagements; very large contracts (100+ pages) would push it higher.
Time constants: 6.0 hours of senior reviewer time per manual review (midpoint of the 4–8 hour range); 0.5 hours of senior reviewer time per automated review, spent only on items the system flagged for human judgment. The pipeline itself runs in ~30 seconds and is not billed against human time.
Closing
We identified a manual process: senior-reviewer-led contract compliance review, 4–8 hours per review, expensive, inconsistent, fatigue-sensitive. We designed an automated solution: a retrieval-augmented multi-step verification pipeline producing reviewer-ready compliance reports with full audit trails, where the senior reviewer remains in the loop on flagged items only. We measured the impact: ~960× faster, more than 90% lower loaded cost per review, more consistent, and demonstrably better at catching subtle classes of error.
The architecture generalizes. Any workflow at a professional services firm where AI-generated artifacts must conform to formal specifications — engagement scope monitoring, methodology adherence reviews, deliverable acceptance, compliance memo drafting — is a candidate for the same approach. The pattern is constant; the specifications and verification layers adapt to the workflow.