Capability debt in the age of AI
A few years ago, I wrote about a problem I kept running into in our work with public sector organisations. I called it capability debt. The idea was borrowed from software engineering: when you cut corners building something, you accumulate 'technical debt' that has to be paid back later. In consulting, we were doing the same thing – leaving organisations with new services and ways of working, but without the underlying capability to sustain them. The consultants depart, and the debt starts compounding quietly in the background.
That problem hasn’t gone away. But something has the potential to make it significantly worse. The AI era we're on the cusp of.
Esteemed writer and thinker Stewart Brand has a useful observation about cities that illustrates his concept of pace layers[^1]. If you compare two maps of downtown Boston, one from 1860 and one from 1960, virtually every building has been replaced. But the roads haven’t moved. The infrastructure that enables everything else, the connections, the boundaries, the flow, those all persist. Brand’s conclusion is that if you want to create lasting change, build the roads, not the buildings.
Most AI adoption in public services right now is building very impressive looking buildings on roads that don’t yet exist.
The debt has compounding interest
The original capability debt argument was essentially about skills and knowledge transfer. It challenged whether we were leaving organisations in a place where they were able to carry the work forward without us. In the AI era, that question becomes three questions simultaneously, and they might just compound on each other in ways that are genuinely new.
Governance debt is accumulating fastest. Organisations are adopting AI tools, often under real pressure to demonstrate innovation, without building the institutional capacity to evaluate, challenge, or govern them. When a procurement decision is made, a vendor’s system goes live, or an organisation wide capability is switched on, somebody needs to be able to ask hard questions about how it works, when it’s wrong, and what happens next. In most organisations we work with, that somebody doesn’t exist yet. The road hasn’t been built.
The evidence is already surfacing across UK government. A report from only 18 months ago suggested at least 55 AI systems were in use across public authorities, yet the official government register listed just nine. This number has since increased ten fold, so begs the question of just what the landscape looks like in reality today. One particular case still exemplifies this failing, where biased algorithms and decisions led to the A-level controversy resulting in 100,000 students having their exam results tampered with based on the perceived affluence or deprivation their postcode signalled historically.
Systems deployed, decisions made, accountability trailing behind. That’s not a technology problem, it's what governance debt looks like at scale.
Data debt isn’t new, but AI puts it under a different kind of stress. We know most public sector organisations have fragmented, inconsistently structured, and poorly governed data. Deploying AI on top of that doesn’t solve the underlying problem, it simply masks it, until something fails in a way that’s visible and consequential.
Dublin City Council recently published a candid account of exactly this challenge. Writing about their efforts to understand land, address, and ownership across the city, Ré Dubhthaigh described how their current approach to data is "restricting our ability to join systems together, understand what is happening and shape a better city." There's no definitive list of who owns what land, ownership boundaries based on buildings that no longer exist, addresses that have become extinct but persist in legal documents. As Dubhthaigh puts it, when is as important as where, and right now, most public sector data systems can't reliably answer either question.
This isn't a Dublin problem. It's the condition of most place-based public services across the UK and Ireland. The dangerous thing about deploying AI on top of this isn't that it fails loudly. It's that it fails quietly, confidently, and at scale, with a risk of producing outputs that look authoritative whilst drawing on foundations that are anything but. You can't build a road on ground you haven't surveyed.
Judgment debt is the most important and the least discussed. Developer and technologist Simon Willison recently observed that "Writing code is cheap now". The barrier to building something with AI has dropped dramatically. He’s right, and it’s significant. But it shifts the critical bottleneck. If producing the building is easy, the thing that matters is knowing whether you should build it, whether it’s working, and when to trust it. That requires human judgment, deep domain expertise, and the organisational confidence to interrogate an algorithm's recommendation, rather than just accept it. In contexts like social care or housing or public health, where the stakes are high and the edge cases are everywhere, that capacity is everything. A design colleague recently shared their personal experience of waiting for a neurodevelopmental assessment for their child which painfully details the human cost of these judgements.
When this capacity atrophies, or when the judgment gets outsourced to the system, you’ve borrowed against something you can’t easily pay back. That's bad debt.
The roads we need to build
None of this is an argument against AI adoption in public services. It’s an argument for building in the right order.
The roads in this context aren’t glamorous, but they are the "mundane magic". They’re data governance frameworks that actually get used. They’re upskilling programmes designed around critical evaluation, not just AI tool adoption. They’re decision making structures that keep human judgment in the loop, in a meaningful way. They’re the internal capacity to commission AI work intelligently, not just adopt it at face value.
The problem is these things take longer to build and establish than a typical procurement cycle allows for. They require sustained investment in people and process rather than technology. They go against the grain of how most transformation programmes are structured, funded and sold within organisations.
We’re sitting with this tension ourselves at TPX. In our AI community of practice, we’re actively exploring what it means to help clients build these roads rather than just helping them commission more buildings. It’s raising harder questions than we expected. Questions about how our engagements are structured, how AI becomes a collaborative force within our practices, what we measure, and what it genuinely means to leave an organisation in better shape than we found it.
The honest answer is that we’re still working it out.
But the direction is clear. AI amplifies everything underneath it. The good infrastructure and the bad. Organisations that invest in the roads first will find AI genuinely transformative. Those that skip straight to the buildings will find themselves, eventually, with a debt they can’t service. Build the roads!
Starting to build the roads
I think Matt Webb's concept of strategic pathfinding[^5] offers a useful starting point here. Writing about AI strategy, he argues that organisations are already in a capability "overhang". Essentially the technology exists and is already reshaping the environment around you, whether you've engaged with it or not. The question isn't whether to act, but how to act in a way that builds lasting capacity and capability rather than just adding another building on unmapped ground.
That framing maps directly onto each of the three debts set out earlier, and might provide a way to begin approaching each of them:
On governance debt: start by making the invisible visible. Before deploying anything, audit what's already in use. Most organisations we work with are surprised by how many AI tools are already embedded in workflows. Often informally, often ungoverned, often as shadow IT! Webb suggests developing simple guidelines that describe when AI use is appropriate, when it isn't, and who to escalate uncertainty to. This isn't a bureaucratic framework, but a shared set of principles to give people permission to act, and clarity about where the edges are. That's a road! In contrast, a governance structure that only exists in a procurement document is a building. Start identifying which of these you're really using across your organisation.
On data debt: a good place to begin is in resisting the temptation to treat a data audit as a precursor to something more exciting. It is the thing. Understanding where your data is fragmented, temporally unreliable, or structurally misaligned, as Dublin City Council is working through openly right now, is foundational work with compounding returns. The organisations that do this unglamorous work first will be the ones who can actually act on AI outputs with confidence rather than anxiety. It might not be exciting, but with some compelling narrative and storytelling wrapped around, it can get the support and recognition it deserves.
Finally on judgment debt: a really transferable idea is that pathfinding isn't for a small isolated AI or data team, it's about making the entire organisation more capable. That means investing in shared learning: running experiments, documenting what works and what doesn't, and building the internal shared understanding to interrogate a process or system's recommendation rather than simply accept it. In public services especially, this critical judgment capacity isn't a nice to have. It's the thing that stops a flawed system running quietly for four years.
None of this is fast or shiny. But it's the work that determines whether AI genuinely transforms public services or just adds a new layer of confident sounding debt on top of the old kind.
[^1]: Brand, S. (1997) How Buildings Learn: What Happens After They're Built
[^2]: Code is cheap – Simon Willison
[^3]: Dublin data challenges – Ré Dubhthaigh
[^4]: I recognise that look - Christine Browne
[^5]: AI Pathfinding – Matt Webb
by KJ