Construction AI Pilot Evaluation: A Stop-or-Scale Evidence Checklist

Read the current construction AI evidence at its stated level

Suffolk and the Massachusetts Institute of Technology published Construction in the Age of AI in September 2026. The paper brings together a preliminary literature review, an industry roundtable, surveys and qualitative input, and a Suffolk model applied retrospectively to a 180,000-square-foot multifamily project completed in 2024. The paper calls its findings directional, describes itself as a methodology statement rather than a definitive causal analysis, and says broader project datasets and controlled or quasi-controlled comparisons are still needed.

Construction Dive independently reported that the case study was a retroactive analysis and that the results were not causal. The publication is a timely reason to build better pilot evidence; its modeled cost, schedule, and return estimates are not achieved results, construction-wide benchmarks, NEXUS results, or a promise for another project.

Use one seven-gate pilot decision record

The seven gates are a NEXUS editorial framework synthesized from the cited methodology, risk-management resources, and product boundaries. Suffolk, MIT, Construction Dive, and NIST did not publish this exact checklist.

A pilot can pass one gate and still fail another. Preserve the evidence and the unresolved condition instead of averaging quality, risk, review effort, and business value into one confidence score.

• 1. Boundary gate: one user task, project context, user group, decision, included and excluded actions, duration, and stop conditions.
• 2. Baseline gate: a frozen comparison window, known records, ordinary workflow, current effort, current error or exception pattern, and material constraints.
• 3. Provenance gate: authorized inputs, source identity and revision, model or service version, configuration, output, actor, time, and retained correction history.
• 4. Review gate: qualified reviewer, sampled ground truth, material corrections, false flags, missed items, uncertainty, review time, and final disposition.
• 5. Risk gate: privacy, security, intellectual property, safety, compliance, commercial, bias, access, retention, and incident controls with named owners.
• 6. Outcome gate: task-specific quality, timeliness, coverage, rework, adoption, transition cost, downstream effect, and evidence quality measured against the baseline.
• 7. Authority gate: a recorded stop, revise, extend, or scale decision with rationale, conditions, responsible owner, expiry, and revalidation trigger.

Gate 1: bound the hypothesis and consequences

Start with a sentence that can be disproved: for a named user and task, using a defined source set, the assisted workflow is expected to improve specified measures without crossing listed risk or authority boundaries. Do not begin with “test AI across the project.”

Name the consequential decisions that remain outside the pilot. Estimates, bid awards, design judgments, code compliance, safety actions, inspections, contract notice, field direction, payment approval, and fund release require their own governing authority and evidence. A pilot may prepare information for those workflows without making the decision.

• Identify the project, phase, work package, geography, contract context, task owner, users, reviewers, and affected parties.
• Record the vendor, model or service, version, configuration, integration path, data location, and any material change during the window.
• Define what the pilot may read, draft, classify, compare, or flag, and what it may not submit, approve, publish, direct, or change.
• Set immediate stop conditions for unsafe advice, unauthorized disclosure, loss of source traceability, material missed items, or attempted action outside the pilot boundary.

Gate 2: freeze a fair baseline before measuring speed

Choose a completed or representative task with source records and a known review outcome, or define a prospective comparison before the work begins. The comparison should use the same task boundary, document set, complexity, reviewer role, and acceptance rule. If those conditions differ, record the mismatch rather than presenting a simple time comparison as causal.

Keep unavailable baseline values as unavailable. A missing error count is not zero, and a historic average from another project is not this project’s baseline. Separate observed values, reviewer estimates, vendor claims, and modeled scenarios.

• Baseline identity: task, source set, revision, sample-selection rule, time window, responsible people, and known outcome.
• Ordinary effort: preparation, execution, review, correction, routing, and downstream follow-up time.
• Quality pattern: material errors, omissions, false flags, unresolved exceptions, rework, and acceptance or rejection.
• Context: project type, phase, delivery method, organization, geography, data completeness, and other conditions that limit comparison.

Gate 3: preserve input, output, and correction provenance

Every material output should point back to the source record actually available to the pilot. Preserve document identity, revision, page or location where practical, access authority, processing time, model or service version, prompt or workflow configuration, generated output, reviewer change, and final disposition.

Do not silently replace an input with a later revision or overwrite a generated answer after review. If the tool, model, system instruction, connector, or source set changes, create a new test revision so the result remains reproducible within the disclosed limits.

Gate 4: measure the work transferred to people

Generation time is only one part of the workflow. Count the time and expertise needed to collect and clean inputs, configure the tool, check sources, correct output, resolve uncertainty, document the decision, and repair downstream mistakes. A faster draft that requires more expert review can still be a poor pilot result.

Use a known-sample review designed before seeing the output. Classify corrections by severity and consequence, not just count. A missed safety, scope, quantity, privacy, or contractual issue is different from a style edit. Keep false positives and false negatives separate and retain the reviewer’s evidence.

• Source coverage: material statements or flags with an openable, correct source reference.
• Material correction rate: outputs changed because the source, interpretation, scope, unit, identity, or status was wrong or incomplete.
• Missed-item rate: known material issues the assisted workflow did not identify within its stated task.
• False-flag rate: issues raised without a source or task basis, with the review cost they created.
• Review burden: qualified-person minutes by preparation, verification, correction, exception routing, and approval stage.
• Residual uncertainty: material questions still unresolved when the output reaches its permitted end state.

Gate 5: treat risk controls as pass or fail

The Suffolk and MIT paper warns that AI output may be incomplete, inaccurate, or based on unreliable project data; may expose proprietary design, commercial, or site information; and may blur accountability. It calls for secure data practices, transparent and auditable AI use, and accountable professional judgment.

Translate those concerns into testable controls. Confirm data authority, least-privilege access, retention and deletion behavior, approved integrations, incident routing, human override, and the system-of-record boundary. If a critical control is absent, do not offset it with a time saving or a high average score.

• No private bid, employee, customer, safety, design, commercial, or project data enters an unapproved service or public output.
• No AI draft or flag is presented as verified fact, professional judgment, approval, instruction, acceptance, payment status, or provider settlement.
• A responsible person can stop the workflow, inspect the source and correction history, and route an incident without depending on the model.
• Affected users know when AI is involved, what the output can and cannot establish, and how to challenge or correct it.

Gate 6: measure outcomes and full transition cost

Choose measures tied to the bounded task. Examples include source coverage, review turnaround, correction severity, exception closure, rework, adoption, and downstream record completeness. Do not pool an estimating pilot, a daily-log pilot, and a schedule pilot into one accuracy or return figure; each has different ground truth, risk, and decision paths.

Include implementation and transition costs: data preparation, integrations, permissions, training, quality assurance, parallel operation, review, incident handling, vendor fees, internal support, and workflow redesign. Record who bears each cost and whether it is one-time, recurring, or expected to change at scale.

Gate 7: record a stop, revise, extend, or scale decision

An authorized owner should compare the evidence with thresholds set before the final review. “Users liked it” or “the demo was fast” is not a scale decision. Preserve dissent, unresolved risk, weak samples, context changes, and any dependence on manual work that will not be available at scale.

• Stop when a critical risk or authority boundary fails, source traceability is lost, or material harm cannot be contained.
• Revise when the use case remains valuable but the scope, data, configuration, control, reviewer path, or measurement design is defective.
• Extend when the pilot is operating as intended but the sample, duration, user coverage, or comparison evidence is still too weak for a scale decision.
• Scale only within the evaluated boundary when quality, risk, review burden, outcome, and cost thresholds pass and accountable owners accept the residual risk.
• Set an expiry and event-triggered revalidation for model, vendor, integration, data, project, regulation, risk, or workflow changes.

Keep domain pilots with their governing workflows

Use this guide for the cross-domain pilot decision record. Use the AI construction bidding guide for takeoff, bid comparison, procurement exceptions, and award boundaries. Use the construction daily-log guide for shift records, field evidence, corrections, and routing into RFI, change, safety, inspection, schedule, or payment workflows.

That separation prevents a generic AI checklist from replacing the qualified review, source standards, contract procedures, safety controls, and decision rights that govern the real task.

Where NEXUS fits the evaluation

NEXUS is currently a beta, AI-assisted construction-operations platform. Its public product facts describe draft plan extraction, takeoff, estimate, scope and risk review, bid and project records, schedules, RFIs, submittals, changes, daily logs, field evidence, approvals, invoices, and reconciliation states.

Those public beta capabilities can support parts of a bounded pilot: authorized source inputs, AI-assisted drafts, bid and project records, field records, approvals, and other human-authorized workflow records. The seven gates are an editorial evaluation framework, not a claim that NEXUS automatically enforces every control or that a NEXUS pilot has produced the outcomes modeled in the Suffolk and MIT report. Qualified and authorized people remain responsible for validation and consequential decisions.