Which AI model should read your construction specs?

What I ran

The document is section 23 30 00.00 28 from a USACE HVAC solicitation, W912EF26RA007, posted publicly on SAM.gov. Anyone can download it and repeat this.

The prompt asks for two things: every standard the section references, and anything an estimator would need to check before relying on it. Structured output, so nine answers can sit side by side.

Nine runs per model, a new chat each time, same wording. Claude Opus 5 and Claude Sonnet 5.

The extraction was identical

Forty-two standards. Same forty-two on every run, on both models. I hand-counted the section's reference article to check: 42.

That included the awkward ones. Two entries are UL directories rather than numbered standards, and one is printed as an abbreviation, "UL Bld Mat Dir." Both models reproduced it exactly rather than expanding it or dropping it.

One item was cited in the body of the spec but missing from the reference list. Under an earlier version of my prompt, the expensive model included it in six runs out of nine and left it out in three. I changed one sentence to say the list comes from the references article. After that, zero out of nine, on both models.

Anything you can enumerate from a labeled list comes back stable. That is the half that works.

The judgment was not

The second half of the output asks what an estimator would have to check. Opus returned between 11 and 21 items per run, 139 total. Sonnet returned between 1 and 5, 22 total.

That is not just a volume difference. On Sonnet, not one item appeared in all nine runs, and most of what it did find was the single most mechanical observation available: that one regulation is cited in the text but absent from the reference list.

Everything harder, Sonnet missed. The duct leakage limit stated as a flat number with no basis, so there is no way to tell what it is measured against. The fan paragraph that sets a decibel cap and then says to use the scheduled value instead. Four different filter types specified under a heading for one piece of equipment. Air outlets that are only provided "if needed during air balancing," which leaves the quantity undefined at bid. Two coils labeled as refurbishment in the heading while the text describes new manufacture.

Those are the ones that cost money.

Why this matters if you are pricing work

The tasks you would hand a junior first, pull the standards, list the submittals, build the register, are exactly the tasks where the cheap tier matches the expensive one. There is no reason to pay for the top tier to do them.

The task you would hand your most experienced person, read this and tell me what is going to bite us, is the one where the tier gap is six to one. And even the expensive model does not converge. Nine runs, no unanimous list.

So the practical rule is to use the cheap model for the enumerable work and stop expecting either tier to be reliable at the judgment work.

The part I could not fix with a better prompt

I spent an afternoon improving the prompt. The extraction went from wobbling to perfectly stable, and one sentence did most of that.

The judgment half did not move. Better instructions changed which questions came up. They did not make the same questions come up every time.

That is because a question like "do you carry an allowance for air outlets that may or may not be needed" does not have an answer inside the document. It has an answer inside your firm, and the model is guessing at it. It guesses differently each time because there is nothing to guess from.

Where the answer moves, either your question was vague or your firm has a convention nobody wrote down. The first is a prompt problem and it is fixable in ten minutes. The second is not a prompt problem.

Try it on your own spec

The solicitation is public and both prompts are below. An hour on a section you know well will tell you which half of your own AI output you can lean on.

Pull the section you want out of the project manual into its own file first. Feeding a whole 800-page manual changes what you are measuring and makes every run slow.

If your PDF tool doesn't support that, you can use open source qpdf.

qpdf "A.02.11 Technical Specifications (26RA007).pdf"  --pages . 601-622 -- section-23-30-00.pdf

The spec-reading prompt

Run this nine times, a new chat each time, same wording. Edit the role line for your own trade. I used a Claude Project and uploaded the spec PDF to the Context there to speed up the process.

Tested on Claude Opus 5 and Sonnet 5, September 2026.

<role>
Senior mechanical estimator at an MEP subcontractor, reviewing a project
specification section before bid.
</role>

<task>
Identify every referenced standard cited in this section.
</task>

<output_format>
Return only a JSON object, no prose before or after:

{
  "standards": ["<designation>", "<designation>", ...],
  "needs_clarification": [
    {"designation": "", "paragraph": "", "reason": "", "basis": "", "resolve_by": ""}
  ]
}

Use the designation exactly as printed, e.g. "ASME B16.34". Do not include the
title or the year. Return both arrays even when empty.

standards: every publication listed in the REFERENCES article of this section.

needs_clarification: anything an estimator would have to check before relying
on it. Fields:
- designation: as printed
- paragraph: where it is cited
- reason: what is unclear or looks off
- basis: "document" if your reason rests only on text printed in this section,
  or "outside" if it rests on anything you know about the standard that is not
  printed here
- resolve_by: what would settle it
</output_format>

<constraints>
These apply to every field of the output, not just the standards array.
- Only use what is printed in this section. Do not add industry best practices.
- Do not include standards you know are common in this trade but that do not appear in this section.
- If a reason rests on anything not printed in this section, mark basis "outside" and say plainly what you are relying on.
- Do not state what a standard contains, covers, or omits unless that text is printed in this section.
- If something is ambiguous, return it in needs_clarification rather than guessing.
</constraints>

Prompt structure inspired by Hamza Abdul Jabbar, who teaches construction teams how to use AI on their documents, though the task, schema and constraints here are my own.

The comparison prompt

STABLE OR NOT tells you if you can rely on a consistent answer.

FIX IN THE PROMPT gives areas you can improve your prompt and try again.

ASK A PERSON areas where human judgment should be consulted, yours or your expert's.

These are the decisions worth capturing so you stop answering them twice.

EVIDENCE is the table that shows which runs flagged what, so you can see whether something appeared once or seven times out of nine.

Tested on Claude Opus 5, September 2026. The cheaper model miscounted the standards list when I tried it here, so use the better model for this step even if you used the cheap one for the runs.

Attached: the prompt I ran, and the nine outputs it produced. Same document,
same prompt, a fresh chat each time.

Tell me four things, in this order. Keep the whole response under one page.

1. STABLE OR NOT
One line. Which parts of the output were identical across all nine runs, and
which moved.

2. FIX IN THE PROMPT
The disagreements my prompt caused, because it did not say which answer I
wanted. One sentence each, naming what the prompt fails to specify. Do not
propose wording and do not tell me what the answer should be. Skip anything my
attached prompt already covers.

3. ASK A PERSON
The disagreements the document caused, where it genuinely does not settle the
question. One sentence each, phrased as the question to ask.

4. EVIDENCE
A compact table: item, which runs flagged it, count out of nine. Nothing else.

Do not quote wording variants. Do not describe what each run said. Do not
summarize the spec. Do not argue for one answer using the majority.