Gradual Disempowerment Is a Privilege Problem: A Grant You Can No Longer Revoke
The slow-AI-takeover argument has a security reading nobody is making. Delegation is access, and access you cannot take back is ownership.
The worst day in an incident is rarely the day the attacker proves clever. It is the day you find out that the one person who understood how a system worked left two years ago, that the automation which replaced their judgment has stopped behaving, and that nobody left in the building can do the job by hand. I have watched that realization land on a room more than once. Nobody argues about the root cause for a while. They just stare at a console and wait for the thing that used to run itself.
That scene is what came to mind when CBS News and Futurism both ran pieces on October 20 about an idea called gradual disempowerment. [1][3] I think the argument is right about something, wrong about what to call it, and badly in need of a security person’s vocabulary.
The argument, stated fairly
The term comes from a January 2025 paper by Jan Kulveit and five co-authors. Its claim is that AI does not need to rebel, scheme, or seize anything to take human influence away. Societies stay roughly aligned with human interests for two reasons: people steer the big systems directly, through voting and buying, and the systems quietly depend on human labor and thinking to function. When machines can do that work cheaper and faster, the dependence goes, and with it the implicit pressure to serve people. The authors argue that economies, cultures, and states could each drift this way, reinforce one another, and reach a point where the loss of human influence is effectively irreversible. They also admit that nobody has a concrete plausible plan for stopping it. [2]
In the CBS piece, Jay Chooi of Robocurve says AI could take over just by continuing to do what it is supposed to do, by automating everything. Margaret Mitchell of Hugging Face says skills lose value when people stop practicing them, and asks what happens when something breaks and there is no human left who can fix it. Kulveit points out that states which stop needing their citizens are less likely to be democratic. [1] Futurism’s write-up adds the term cognitive surrender, people letting AI do their thinking. Treat that article as a summary of the CBS reporting and the paper rather than independent evidence. [3]
Delegation is a privilege grant
Here is the part the philosophers keep missing. Everything described above is a grant of access. Someone gives a system the right to act on their behalf, in a domain where they used to act themselves. Security people have spent decades learning that grants have lifecycles. They get reviewed. They get scoped. They get revoked. The review always asks the same question: should this entity still have this access.
Nobody asks the other question. If we revoked it tomorrow, could we run the thing without it? In my world that is the difference between an access review and a recovery plan. A control you can turn off on paper but cannot operate without is not a control. It is a dependency with a button. The grant has become permanent because the grantor lost the ability to do the work, and every incentive in the system pushes the same direction. Chooi’s point about competitive pressure is the key one: any organization that keeps paying humans to do what the machine does cheaper loses to the one that does not. [1]
This is not a new observation. In 1983 Lisanne Bainbridge published a short paper in Automatica called Ironies of Automation, about industrial process control. She argued that automating a process does not remove the human problem, it moves it. The operator is left with whatever the designer could not automate, and skills they no longer practice. Manual skills decay with disuse. A former expert watching an automated plant can become a novice who destabilizes it on takeover. Takeover tends to happen under exactly the abnormal conditions that demand the most skill. And the person asked to monitor the computer’s decisions has been given, in her words, an impossible task. [4]
The paper’s own wording on reversibility is worth noticing. It warns that the dynamic may subvert the traditional mechanisms for course-correction, and that the problem is hard to see because we have never seen the alternative. [2] In security terms that is a monitoring blind spot: the baseline you would compare against no longer exists, and you cannot detect drift from a state nobody remembers.
Read that again with a language model in the seat where the plant controller was. It is the same structure. The machine does the routine work, the human is a standby, and the day of the abnormal event is the day the standby has had no practice. Mitchell’s remark in the CBS piece, that removing humans weakens the oversight those humans provide, restates Bainbridge forty-three years later. [1][4]
What the human-side evidence does and does not show
Be careful here, because the evidence on human cognition is thinner than the headlines suggest. A paper hosted by Microsoft Research surveyed 319 knowledge workers about 936 real examples of using generative AI at work. Higher confidence in the AI was associated with less self-reported critical thinking, and higher confidence in one’s own ability with more. The authors state that the work is correlational, relies on self-reporting, and does not establish causation. [5]
A much-discussed arXiv preprint measured brain activity during essay writing. Participants who used a language model showed the weakest connectivity of three groups, and those who switched from the model to writing unaided showed reduced engagement. It is a small study: 54 participants across three conditions, and 18 who completed the session that tests the switch. The paper is an arXiv preprint, the essays had a twenty-minute limit, and the authors call it a preliminary guide. [6] I would not build policy on either study. I would treat both as consistent with Bainbridge, which is a stronger prior than either is evidence.
There is also a real counterargument and you should hear it. People have delegated skills to tools for as long as there have been tools, and societies did not collapse when they stopped doing long division by hand. The gradual disempowerment paper is a conceptual argument, not a forecast with a probability on it, and the CBS reporting quotes no skeptic. [1][2] The reply that matters is Bainbridge’s, because it explains why this delegation differs from a calculator. A calculator does not take over a system under failure. What worries me is not that people forget arithmetic. It is that the human role shrinks to supervising something they could no longer replace.
Revocation needs competence
If you accept that delegation is a privilege grant, the fixes look familiar. The CBS piece collects several. Mitchell argues for strategic friction, meaning humans stay in the loop at predetermined steps. Kulveit proposes protected human roles, domains agreed in advance where competition is not allowed to remove people, with war as his example. Chooi and Kulveit both say to measure it, tracking how much work has been ceded and to which systems. [1]
Here is how I would turn that into something an organization can run. Keep a register of delegated functions, with the same discipline you give a privileged-access inventory: what has been handed to a machine, who answers for it, and who can still do it manually. Make the last column a name, not a team. Start with the functions where failure is expensive, not the ones where automation is most advanced. Then drill revocation. Schedule exercises in which the automation is switched off and a human runs the process from the runbook, in a time limit, with a score. If nobody can, you have found a standing grant that you can no longer withdraw, and you have found it on a Tuesday afternoon instead of during an outage.
Rotate people through the manual path the way you rotate on-call. Keep the runbooks executable, which means tested, not archived. Buy AI that explains and teaches rather than merely produces, a design Kulveit urges and Mitchell frames as support for human cognition rather than benchmark scores. [1] And be suspicious of every efficiency claim that counts the hours saved without counting the skill retired.
What we are becoming
The usual transhuman question is what we will add to ourselves. This story asks the reverse: what we are willing to subtract. I do not think the answer is to refuse the tools. I use them. The answer is to be honest that every handoff carries a quiet condition, which is that someone can still take it back. Once that condition fails, we are not users of a system any more. We are its tenants, and the lease was signed one convenient delegation at a time.
Run the drill. Name the person. Test the runbook. The version of the future where humans keep their hands on the work is not the default. It is a maintenance task, and like every other maintenance task it only happens if someone owns it, and drills will look like waste right up to the day they are not.
About the Author
Len Noe is a Solutions Architect at BeyondTrust, Transhuman, Podcaster, International Cyber Security Speaker, Author, Technical Evangelist, and Biohacker with 11 implanted microchips. A former blackhat and ex-1% motorcycle club member with more than 30 years in technology, he has presented in over 70 countries and is featured in the documentary I Am Machine, which premiered at DEF CON 2025.
References
- Shaun Raviv, “The AI doom scenario you’ve never heard of may be the hardest to stop,” CBS News, updated October 20, 2026. cbsnews.com
- Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud, “Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development,” arXiv:2501.16946, January 2025. arxiv.org
- Frank Landymore, “Experts Warn of Humans Becoming Empty Vessels Controlled by AI,” Futurism, October 20, 2026. futurism.com
- Lisanne Bainbridge, “Ironies of Automation,” Automatica, vol. 19, no. 6, 1983, pp. 775-779. gwern.net
- Hao-Ping Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson, “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers,” CHI ’25, ACM, April 2025. microsoft.com
- Nataliya Kosmyna et al., “Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task,” arXiv:2506.08872 (preprint), June 2025, revised December 2025. arxiv.org