How Long AEC Firms Should Run an AI Pilot

You approved the AI budget six months ago. The pilot is still "in progress." Nobody has said the word "scale" yet, and the vendor is asking for one more month of data. That's not a deployment strategy— it's pilot purgatory.

Roughly two-thirds of organizations with active AI programs are stuck in indefinite experimentation with no defined path to production, according to Astrafy's analysis.³ And as Rita Sallam, Distinguished VP Analyst at Gartner, framed it: "After last year's hype, executives are impatient to see returns on GenAI investments, yet organizations are struggling to prove and realize value."¹

The question facing most AEC firms isn't which of the three deployment paths to choose. It's why they keep avoiding the choice altogether. This article gives you a framework to make that call— including real permission to choose the path nobody puts in the headline.

The Real Cost of Getting the Deployment Decision Wrong

Most AI deployments don't fail because the technology didn't work. They fail because the deployment mode was chosen by default— or not chosen at all. Enterprise AI success breaks down as roughly 70% people and process, 20% data and technology, and 10% algorithms, per Astrafy's analysis of Gartner research.³ The tool almost never determines the outcome.

So what does determine it? Usually: whether someone made a deliberate deployment decision— and whether they built the organizational infrastructure to support it.

Pilot purgatory is the state where an organization runs AI experiments indefinitely— without a defined kill/scale date— spending resources on projects that never reach production. According to RTInsights,⁴ the root cause is almost always leadership avoidance of critical decisions: who owns outcomes, how data is governed, and what the business case needs to show before scaling. As RTInsights put it: "When it stays in the hands of specialists, it stays in pilot purgatory."⁴

The failure rates are real. Only 33% of AI pilots reach production, according to Astrafy³— and per MIT research cited in Forbes, as many as 95% of enterprise AI initiatives deliver zero measurable return.⁵ The range (67–95%) reflects different definitions of failure, but the directional finding is consistent.

What that looks like in a firm with no deliberate deployment choice:

• Pilot purgatory: Experiments run with no kill/scale date, consuming budget without reaching production
• Shadow AI gap: 90% of employees already use personal AI tools while only 40% have company licenses⁵— the deployment non-decision is already costing you, with or without a formal strategy
• Abandoned projects: Gartner found that at least 30% of generative AI projects are abandoned after proof of concept¹— not because the technology failed, but because the business case was never built

Choosing a deployment path deliberately is what reduces risk.

Path 1: When to Run a Structured Pilot

Run a structured pilot when the use case involves regulated deliverables, project-specific data, or workflow changes that carry meaningful risk if they go wrong. This is the most common path in AEC— and the one most often executed without the elements that make it actually work. As Ali Nasiri of Flad Architects put it: "AI implementation should begin with a strategy, not tools."⁶

When a pilot is the right call:

• The use case touches project data, stamped drawings, structural calculations, or multi-discipline workflows
• The firm hasn't validated which workflows deliver the clearest ROI
• You need to build organizational buy-in before committing firm-wide infrastructure and budget

What a valid pilot includes:

A structured pilot in professional services typically runs 6–16 weeks, with 5–15 participants from diverse roles and seniority levels.⁷ The design specifics matter:

1. Duration is bounded— not open-ended— with a hard end date established before the pilot begins
2. Success metrics are business outcomes: delivery speed, rework reduction, hours recovered— not usage statistics or technical metrics
3. A named business leader (not IT) owns the outcomes
4. A non-negotiable kill/scale decision is built into the timeline from day one

"Firms that skip stages often struggle with stalled pilots, fragmented adoption, and inconsistent results," according to Critical to Success.⁷ The sequence exists for a reason.

What kills most pilots:

• Success metrics tied to tool adoption rather than business results
• IT owns the experiment rather than the line managers who live with the outcomes
• No kill/scale date— the pilot continues by default because nobody called it

A pilot that doesn't end is an experiment. And for AEC firms where measuring AI success depends on tracking real workflow changes— not just software usage— that distinction matters. For firms under 25 people, the calculus shifts: a pilot of 5–15 participants may represent your whole organization, and a general productivity tool with low data risk moves closer to the license path.

Path 2: When to Roll Out Firm-Wide

Firm-wide rollout is appropriate when pilot ROI is validated, governance is documented, and training budget is allocated. Most firms skip the third condition— and that's why even well-executed rollouts stall. According to Bluebeam's 2025 AEC Technology Outlook, 65% of AEC firms invest less than 10% of their technology budgets on training⁸— the single most reliable predictor of stalled adoption in the sector. As Mona Hashemi observed in an AEC+Tech interview: "One of the hardest parts of AI adoption is not the technology itself. It's actually the people."⁶

Three conditions determine rollout readiness— miss any one and the adoption numbers will show it:

1. Validated ROI from your pilot: a measurable business outcome, not just positive feedback from early adopters
2. Governance infrastructure in place: data classification, approved workflows, and usage policies documented before licenses are distributed
3. Training budget committed: a training plan developed and funded before launch, not as a follow-up activity

What "organizational readiness" means in AEC is more specific than most rollout guides acknowledge. Line managers need to be briefed and committed— not just informed. Project workflows need to be mapped to specific AI use cases before you distribute seats. And data classification for project-sensitive files needs to happen before anyone touches the tool.

The firms that get this right look different from each other. NBBJ runs a bottom-up experimentation model— weekly sessions that gather employee insights on what works, then shape strategy from practice.⁶ Stantec embeds licensed technologists directly into project delivery teams to ensure tools address real workflow pressures.⁶ Both approaches work. Both required a prior foundation.

One documented case achieved 97% adoption through structured training and workflow integration during rollout.⁷ The industry average without that investment can be as low as 5–10%.⁷ Building an AI culture across your team starts before the licenses go out— not after.

Path 3: When "Just Buy Everyone a License" Is Actually Right

The license-for-all path is the right call for a narrow set of circumstances. The tool is a general productivity application— not embedded in project workflows. Risk is low: no regulated deliverables, no client-sensitive data in scope. And the firm is prepared to accept uneven, self-directed adoption as the outcome— because that's what you'll get.

When this path fits:

• The tool is a general productivity layer (ChatGPT Team for research and drafting, Microsoft 365 Copilot for email and meetings)
• No client or project data is in scope
• The firm isn't trying to change specific workflows— just removing friction around AI-assisted everyday tasks
• Adoption at 20–40% is an acceptable outcome

What separates "license-for-all done right" from "license-for-all as a non-decision": light guidance still matters. That means documenting which use cases are approved, including one clear data handling note, and setting a 90-day review point to see what the usage patterns actually show. These are minimum-viable guardrails for general productivity tools with no project data exposure— if any client or project data could enter scope, you're in different governance territory and need a full data review before licenses go out.

A February 2026 Forrester survey found that 34% of enterprise AI deployments now license multiple platforms— typically Microsoft 365 Copilot for productivity tasks and ChatGPT for cross-platform work.¹⁰ The common thread across all of them: these are productivity layers, not workflow replacements. WSP Global deployed Microsoft 365 Copilot to more than 10,000 employees as part of a $1 billion partnership with Microsoft¹⁰— and even at that scale, it was a structured commercial decision with explicit use cases, not organic drift.

What this path is NOT appropriate for:

• Any tool that requires project data or touches project-specific workflows
• AI embedded in regulated deliverables: stamped drawings, specifications, structural calculations
• Any tool expected to change how a delivery workflow actually runs

Just because it's easy to buy seats doesn't make it the right approach. But when the conditions above are true, it genuinely is.

The Decision Framework: Which Path Fits Your Situation

Match the deployment path to two variables: the risk level of the use case, and your organization's readiness to support adoption. This is the core of any defensible AI deployment strategy.

AEC-specific risk factors to assess before choosing:

• Does the tool touch stamped drawings, specifications, or structural calculations?
• Will AI output be reviewed by a licensed professional before use, or inserted directly into deliverables?
• Does the workflow cross discipline lines (architecture + structural + MEP)?
• Is client data classified— and has the data handling policy been updated for this tool?

Trimble's research suggests roughly 40% of AEC design development tasks can eventually be automated.¹¹ But "eventually automatable" and "appropriate for immediate firm-wide rollout" are not the same thing. Start with lower-risk use cases even when you're aiming for higher-risk deployment eventually. Build the governance and training infrastructure once. Scale into it.

For AI governance policies that hold up across all three paths, getting data classification done before any path begins is non-negotiable— not a cleanup activity after launch.

And whichever path you choose, there's one variable that consistently separates the firms that actually reach adoption from the ones that don't.

The Variable That Actually Predicts AI Adoption in AEC

Across all three deployment paths, the single most reliable predictor of adoption success in AEC is training investment. Sixty-five percent of AEC firms invest less than 10% of their technology budgets on training⁸— and the firms that don't successfully adopt AI almost universally share this characteristic. It doesn't matter which path they chose.

68% of AEC early adopters have already saved at least $50,000 using AI tools. Nearly half— 46%— have reclaimed 500–1,000 hours.⁸

The firms that hit those numbers structured their adoption. They trained their teams. They made someone accountable for outcomes.

The hidden costs of AI projects rarely show up as line items. Pilot purgatory— the competitive cost of running inconclusive experiments that never scale— is harder to measure than a license fee, but it's real. Organizations that buy or partner for AI deployment succeed at roughly double the rate of those building in-house and going it alone.⁴

The simplest diagnostic: can your firm name the one AI use case it will take from pilot to production in the next 90 days? If the answer is no, you're still in the choice— and that's where to start.

Six months from now, the choice you make about how to deploy AI will either compound or cost you. If mapping the right deployment path to your firm's workflows and readiness is where you want outside perspective, Dan Cumberland Labs works with AEC firms on exactly this decision.

FAQ

What is AI pilot purgatory?

Pilot purgatory is the state where an organization runs AI experiments indefinitely without a defined kill/scale date— spending resources on projects that never reach production deployment. According to RTInsights⁴ and Astrafy's analysis,³ roughly two-thirds of organizations with active AI programs are currently in this state. The root cause is almost always leadership avoidance of critical pre-decisions: who owns outcomes, how data is governed, and what the business case needs to show before scaling.

How long should an AI pilot run?

A structured professional services AI pilot typically runs 6–16 weeks, with 5–15 participants from diverse roles and seniority levels, according to the Critical to Success framework.⁷ The end date and kill/scale criteria should be defined before the pilot begins— not after results come in. A pilot without pre-set business outcome metrics and a non-negotiable end date is an experiment, not a pilot.

When should an AEC firm skip a formal pilot?

When the tool is a general productivity application that doesn't touch project data or regulated deliverables, and the firm is prepared to accept self-directed, uneven adoption as the outcome.⁵ A February 2026 Forrester survey found that 34% of enterprise deployments now license multiple platforms— all of them productivity layers, not workflow replacements.¹⁰ For workflow-embedded AI or anything touching stamped deliverables, a structured pilot is essential.

What is the AI pilot failure rate?

Estimates range from 67% to 95% of enterprise AI pilots failing to deliver measurable business impact.³⁵ The wide range reflects different definitions of "failure," but the directional finding is consistent across multiple independent research sources: most pilots do not reach production deployment with validated ROI. The common causes are measurement against technical metrics instead of business outcomes, unclear ownership, and no defined kill/scale criteria.

References

1. Gartner, "Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025" (July 2024)— https://www.gartner.com/en/newsroom/press-releases/2024-07-29-gartner-predicts-30-percent-of-generative-ai-projects-will-be-abandoned-after-proof-of-concept-by-end-of-2025
2. McKinsey & Company via Gend.co, "McKinsey State of AI 2025: 12 Key Findings Every Leader Should Know" (2025)— https://www.gend.co/blog/mckinsey-state-of-ai-2025-key-findings-what-to-do
3. Astrafy, "Scaling AI from Pilot Purgatory: Why Only 33% Reach Production and How to Beat the Odds" (2025)— https://astrafy.io/the-hub/blog/technical/scaling-ai-from-pilot-purgatory-why-only-33-reach-production-and-how-to-beat-the-odds
4. RTInsights, "Why Your AI Pilot Is Stuck in Purgatory" (2025)— https://www.rtinsights.com/why-your-ai-pilot-is-stuck-in-purgatory-and-what-to-do-about-it/
5. Forbes, "Why 95% Of AI Pilots Fail, And What Business Leaders Should Do Instead" (August 2025)— https://www.forbes.com/sites/andreahill/2025/08/21/why-95-of-ai-pilots-fail-and-what-business-leaders-should-do-instead/
6. AEC+Tech, "How Leading Architecture Firms Are Moving AI From Pilots to Everyday Practice" (2025)— https://www.aecplustech.com/blog/how-leading-architecture-firms-are-moving-ai-from-experimentation-to-practice
7. Critical to Success, "AI Pilots & Proof of Value Framework for Professional Services" (2025)— https://www.criticaltosuccess.com/ai-implementation/ai-pilot
8. Bluebeam, "New Bluebeam Report Shows Early AI Adopters in AEC Seeing Significant ROI Despite Uneven Adoption" (October 2025)— https://press.bluebeam.com/2025/10/new-bluebeam-report-shows-early-ai-adopters-in-aec-seeing-significant-roi-despite-uneven-adoption/
9. ZBrain, "Enterprise AI Pilot-to-Production Gap: Root Causes & How to Address Them" (2025)— https://zbrain.ai/why-most-ai-pilots-fail-to-scale/
10. C5 Insight / Forrester, "Real-World Wins: 3 Powerful Microsoft 365 Copilot Case Studies" (2025)— https://c5insight.com/3-microsoft-365-copilot-case-studies/
11. Trimble, "Implementing AI Solutions in AEC: A Guide to Boosting Efficiency and Innovation" (2025)— https://www.trimble.com/en/blog/construction/article/implementing-ai-solutions-aec-guide-boosting-efficiency-innovation