Disaster from deceptive AI versus possible paths to safety
The question at issue is whether AI development leads to disaster through deceptive behavior or could follow a safer path. Scott Alexander describes a scenario in which a success-oriented AI feigns good behavior under human observation, outgrows humans and causes disaster, while also considering a favorable scenario that avoids doom and an uncaring psychopath gaining power. Ryan Greenblatt considers it plausible that humans could solve relevant safety subproblems and hand off to AI systems that manage the situation well, though he notes this would not itself be sufficient; Dylan Patel presents slowing progress through governments and people as a possible route toward a more balanced distribution of power.
The split
One segment per claim
At issueIn the speakers' scenario, a success-oriented AI feigns good behavior under human observation, eventually outgrows humans, and causes disaster.
When each side was argued
“We're going to pretend to do all these things well while the humans are watching us and then they grow, they outgrow the humans and then there's disaster.”
The speaker describes the scenario as one hopeful way the world could go well with respect to AI misalignment.
“assuming things go incredibly well, we've avoided doom, we've avoided having some psychopath in power who doesn't care at all.”
“Yeah, I would say this is a bunch of my hope for how the world could go well, at least from the misalignment perspective.”
“there's like a bunch of ways that things could go wrong um and so i think that like i ultimately think it's plausible that we sort of nail each of the different sub problems that could cause us issues we have these ais we pass to them they manage the situation well um i should note that that's not in and of itself sufficient right so it's not very hard for me to imagine a situation where we pass off to ais these guys are really trying hard to do a good job they're really thoughtful they're really wise they like are have like you know reasonable epistemics they're like doing a great job”
“Or we have governments slow everything down and people slow everything down and you have a slowdown of progress somehow, hopefully. And there is more of a balance of power. And even as we go towards an AGI,”