Anonymized portfolio case study. Client, employer, project, manufacturer, product, and pricing details have been removed. The validation results and control decisions are real.
Executive result
I built an automated pricing workflow for construction quotations, then tested every matching path against realized historical prices before allowing it to control client-facing totals. The testing exposed an unreliable ambiguous-matching path and flagged a material incorrect line; human review had flagged the same line independently, so it was never a risk to the submitted quote. I removed that path from automatic pricing, retained the validated exact-match workflow, and routed uncertain items to human review.
The result was a defensible boundary between what the system could price automatically and what still required professional judgment.
The pricing risk
The workflow needed to match free-text line descriptions from takeoff exports against a catalog of previously realized prices. Exact descriptions could be matched reliably. Ambiguous descriptions were harder because field language changes between jobs, abbreviations drift, and one description may reference a related product rather than the item being priced.
A fuzzy match could therefore look reasonable while pointing to the wrong catalog item. Once that result entered a quote total, a plausible suggestion became a commercial decision.
The validation approach
I built a QA harness that replayed known description and price pairs through the matching logic and compared each result with the price actually realized. The test was not whether a match looked plausible. It was whether the automation selected the correct priced item.
I evaluated the ambiguous fuzzy path separately from the deterministic exact-match path so that a failure in one would not discredit the parts of the workflow that had earned trust.
What the testing found
- The fuzzy path was wrong in 63 of 66 cases in which it fired.
- A stricter version was correct only once in 21 attempts.
- The worst mismatch exceeded 1,000 times the reference value.
- On a live quote, one phantom line represented approximately 40% of the reported total.
- The exact-match path validated 20 of 20 reference lines and 91 of 98 high-confidence lines on an unrelated job the system had not previously seen.
The fuzzy failure was structural, deeper than any tuning could fix. Descriptions can mention the item a component attaches to rather than the component itself. Text similarity alone cannot reliably distinguish those relationships.
The control decision
I removed fuzzy matches from automatic quote totals before the live quote was submitted. The revised workflow follows four rules:
- Exact validated matches may populate a price.
- Ambiguous matches may identify where a reviewer should look.
- Uncertain results have no automatic dollar impact.
- A responsible person approves every exception before submission.
This preserved the useful automation where confidence had been earned, and kept its least reliable path away from any client-facing pricing decision.
Why it matters
Construction pricing errors do not need to look absurd to be dangerous. The hardest failures are plausible, internally consistent, and buried inside a larger quote. Effective automation must therefore be tested against real outcomes, separated by confidence and failure mode, and designed to fail safely when certainty breaks.
The same discipline applies to estimating QA, document extraction, schedule generation, specification review, and other AI-assisted workflows that influence commercial or technical deliverables.
Automation earns authority only after it survives validation against real outcomes. Until then, it surfaces evidence and a person makes the decision.
Josiah Bujanda | AI and Construction Technology Consultant