The frontier questions are not settled, and I do not think they can be settled from inside one operating reality. What an institution is willing to let a system decide depends on the institution: its regulator, its history of being wrong, its appetite for reversibility, the consequences of failure, and the seniority of the person who ultimately signs. I have one vantage point on that. It is not enough.
So the work ahead is mostly other people's evidence. I am asking a small number of executives the same question and keeping careful track of how their answers differ. Those differences are not something to average away. They may be where the architecture actually lives.
What I am asking
How does your organization decide what a system is allowed to change?
Not what it can do. What it is permitted to do, who decided, and what evidence exists if someone asks.
Not what it can do. What it is permitted to do, who granted that authority, under what conditions, and what evidence exists when someone later asks why.
I have encountered versions of this question for more than two decades.
It began for me in the complexity of Yahoo!, where control had to work across internet-scale systems, distributed ownership, continuous change, and decisions whose effects could travel far beyond where they originated. At Xerox PARC, that question moved closer to machine autonomy as we experimented with large-scale machine learning and autonomous execution through the Big Data Foundry, well before AI became the dominant framing it is today.
Financial crime and risk made the problem explicit. Regulation turned authority, accountability, provenance, and evidence into requirements rather than design preferences. A system could not simply produce an answer. You had to know who had authorized the decision, under what policy, and with what evidence.
Now, through the Wellzai venture thesis, we are exploring trust infrastructure for verification and assurance in food and health, where uncertainty is greater, feedback loops are longer, and many consequences are less reversible.
Across these environments, one thing has become increasingly clear to me: the differences between institutions may not be noise around a universal governance model.
They may be the architecture.
So I am beginning a series of forty-five-minute conversations with people who have actually had to decide when a system should be allowed to act.
I want to understand whether there is a common structure beneath the variation, or whether machine authority must ultimately be derived from the operating reality of each institution.
There is nothing being sold and nothing to demo. What I learn will be written back out, shared with contributors before publication, and attributed only with explicit permission.
The question underneath it all is simple:
When a machine can act, what gives it the authority to do so?
What would change the argument
The inquiry is useful only if the evidence can force a different conclusion.
The institutional differences collapse into a small common patternIf regulation, history, organizational design, and risk appetite mostly change the vocabulary rather than the underlying authority structure, then my current emphasis on institution-specific architecture is overstated.
Existing controls already provide a sufficient runtime answerIf identity, policy, workflow, audit, and control systems can already show who granted permission, for what action, under what evidence, and with what recovery path, then the gap may be integration and operating discipline rather than new architecture.
Consequence and reversibility explain more than intelligenceIf the permission boundary consistently follows the reversibility and blast radius of an action rather than the sophistication of the model performing it, the governing frame should move from model capability toward consequence classes.