What an AI Center of Excellence Does (and Does Not Do)
An AI CoE exists because enterprise AI fails on coordination, not on technology. Gartner projects at least 30% of generative AI projects are abandoned after proof of concept (Source: Gartner, July 2024), and McKinsey reports more than 80% of organizations see no tangible enterprise-EBIT impact from generative AI, with workflow redesign the biggest determinant of whether AI moves the bottom line (Source: McKinsey, The State of AI, March 2025). Neither is a model problem. Both follow from every business unit running its own pilot, against its own rules, on its own data, with nobody accountable for the portfolio.
The CoE is the standing answer: a small, permanent team with a written mandate to decide what the enterprise builds, under what rules, on what platform, with which skills. BCG's 10-20-70 finding — roughly 10% of AI outcomes from algorithms, 20% from technology and data, 70% from people and process — is why this is an operating-model job rather than an engineering one. Iternal covers the split in the 10-20-70 rule.
The four things a CoE owns
Owns the intake queue
Every AI idea enters one queue, is scored the same way, and comes back funded, deferred or rejected with an owner.
Owns the rules
Acceptable use, data classification, model approval, human-review thresholds and incident handling, written once and applied everywhere.
Owns capability
Role-based literacy, a champion in each business unit, and a pattern library so the second project costs less than the first.
Owns readiness
The approved platform, the data pipelines behind it, the security posture, and the evaluation harness every use case must pass.
The three things it should never own
It does not build every use case
Business units build. The CoE sets the pattern, holds the gate and stays out of delivery, or it becomes the bottleneck it exists to remove.
It is not a second IT department
No parallel infrastructure budget, no shadow roadmap. It works through the platform team, not around it.
It is not a monthly review meeting
A committee that reviews slides and approves nothing is the most common failure mode. Decision rights and a clock are what make it a CoE.
The AI CoE Charter: Mandate, Scope and Decision Rights
Nothing about a CoE works until an executive signs a charter. It runs to one or two pages and is the only artifact that converts a working group into a body with authority. Without it the group recommends but cannot decide, and the queue silently becomes a backlog. A workable charter answers seven questions in writing:
- Mandate. One sentence on what the CoE is accountable for — typically "every AI system touching company data or customers."
- Scope boundaries. What is explicitly out: embedded AI in purchased software, personal productivity tools under a stated threshold, research that never touches production data.
- Decision rights. What the CoE decides alone (platform, model approval, acceptable use), what it recommends to the council (funding, headcount, risk acceptance), and what stays with the business unit.
- The decision clock. A published turnaround — ten business days from intake to decision is common — so teams stop routing around the CoE.
- Escalation path. Who breaks a tie, and how fast.
- Funding model. Central budget for the CoE, business-unit budget for delivery. Mixing them turns a CoE into a build shop.
- Review date. A twelve-month review with defined success measures, so the charter renews on evidence rather than inertia.
Compliance obligations land here too. The NIST AI Risk Management Framework 1.0 organizes AI risk work into four functions — GOVERN, MAP, MEASURE and MANAGE — and GOVERN is, in practice, a description of what a CoE charter establishes. For organizations in scope of the EU AI Act, Article 4's AI literacy obligation has applied since 2 February 2025 and belongs to the literacy work stream (EU AI Act Article 4 literacy).