Case study · AI implementation
AI Report Reviewer
An AI review engine that reads a home inspector's real reports and returns them in his own scorecard, grounded in his own standards.
/ context
He tried to build it himself, and it drifted
The owner put hundreds of hours into building this with ChatGPT and Gemini. The DIY versions drifted. The rules got lost in one long chat and the model started improvising. That is how bolt-on AI usually ends.
/ the build
It reads his real reports and answers in his own scorecard
BasisWeb built the v1 engine. It reads his real inspection reports end to end and returns the review in his own scorecard format, the layout he used to fill in by hand at night.
His own standards documents are wired in as the grounding, so it flags things the way he does, not the way a generic model would.
/ why it holds
A prompt is not a system
The rules live outside the model. Each check hands the AI only the slice of his standards that one question needs, so it cannot lose the plot the way a chat window does.
That is the difference between a system and a clever prompt, and it is why this one holds where his DIY attempts wandered off.
/ the number we will not publish
Ask a vendor what they tested their number on
An accuracy number quoted today would look great and mean nothing, because the reports we can test against already contain the owner's corrections. Grading against your own answer key is not a test.
We publish a catch rate only after testing against reports the way inspectors first submitted them. When a vendor hands you a suspiciously clean number, ask them what they tested it on.
/ status ledger
- RunningThe v1 engine, end to end on his real reports
- Being provenCatch rate, measured the honest way
No accuracy number gets published until it is measured against reports the way inspectors first submitted them. Not before.