Boudreaux is a policy researcher at RAND. His research focuses on navigating the transition to transformative AI. He teamed up with RAND mathematician Alvin Moon to model how AI could erode human agency over time—and what humans can do to stay in control.

Agency “is the capacity that lets us set and pursue every other value that we care about,” Boudreaux said. “And the danger is that there might not be some clear, dramatic moment when we lose it.”

What would losing human agency to AI actually look like?

It wouldn’t happen all at once, and it might not even feel like a loss. It could feel like a convenience, a relief. Our decisions might seem to be getting easier. But over time, we would reach a threshold where the options that are presented to us have all been shaped by AI. And then, really, humans would no longer be in charge.

How do you see this happening?

We identified three mechanisms by which AI could erode human agency. The first is disenfranchisement, where humans are simply automated out of decisions. The second is AI enfranchisement, where AIs effectively become members of the deciding groups.

But the third mechanism is really the most subtle. We call it agenda control. The AI doesn’t cast a vote itself, but it shapes what options the humans see. It would be practically invisible. The options the AI excludes are just not there anymore, so humans wouldn’t even know about them.

What are the warning signs that we might be heading toward that kind of tipping point?

What’s most important here is the trajectory, the trends over time. Every individual move to delegate a decision to AI might make perfect sense. But thousands or millions of these very reasonable decisions add up to a situation in which no human is really deciding anything.

So you might look at, how often do humans override an AI recommendation? How much time and space do humans have to make decisions? What is the scope of an AI’s decisionmaking authority—is it handling one task or many? Those are the trends we want to measure.

Let’s get into that, because your real contribution here was to model these trends mathematically. What was your approach?

We were focused on collective agency, how we as humans make decisions together. We built our model on social choice theory, which is the mathematics of how groups come together and make decisions. Then we started to add AI to those decisionmaking groups. We could look at how many humans are in these decisive coalitions, how many AIs, and how that changes over time. We could model these little perturbations and see what happens. And in that way, we found that we could begin to track the erosion of human agency.

Your modeling pointed to an end state, where humans have lost agency and cannot get it back. How would this become irreversible?

There are a few ways. First, in this hypothetical end state, the AIs are in control. They would have to decide to give power back to humans, and that might not happen. Second, we’d be much more dependent on AIs. Even if there were an opportunity to take back control, we might not have the skills or expertise anymore to do it. And finally, once AIs are embedded in all of these different decision-making structures, it would just be very costly and complex to remove them.

How can the government and other groups use your modeling to avoid a future like that?

Whenever a new AI model comes out, there are all these different assessments that happen. Could it contribute to a cyberattack? Does it increase biological risks? But we haven’t had any way to assess what happens to human agency when we deploy these models in different decisionmaking contexts. We’re working now to develop what we call agency audits. The government or companies and researchers could use these audits to assess what might happen to human decisionmaking with each new model that comes out.

Has this research changed how you personally use AI?

You know, in many ways, I feel like my agency has grown with AI. I can code up apps now or build simulations, things I never thought I’d be able to do. But I try to be much more deliberate about how I use it. I lay out my own ideas first, before I ask the model anything, because I worry that whatever it hands me, I’ll tend to accept. I sometimes treat the very convenience as a warning sign. The smoother it feels to just go with a recommendation, the more I think it might be part of the problem.

All of this is a practice that I definitely haven’t perfected. But the idea is that maybe I can use AI and get the benefits, while also preserving my own capacity to set and act on my values.

Go to Original – rand.org