The tracked speakers share an expectation or concern that AI could concentrate economic resources and political power, though they describe different possible outcomes.
“And so there's just this worry that you have models which will basically consolidate all businesses in the world. or at least all current businesses in the world, or at least all current white-collar businesses in the world,”
“So and if anything, as they get better, it's going to get harder to discern when they've made these mistakes, even though they're going to keep making them.”
“And hopefully, you know, different societies just like now with their eyes will align their eyes to a different one. So we have not just a monoculture of. alignment”
The tracked speakers share an expectation that providing safety guarantees or assurances for robots will be difficult.
“but the exact same problem is going to be someone's going to develop a robot, someone's going to need some promises. they're going to struggle to make the promises.”
“That one of the big challenges is having doing effectively a sensing part in a reasonable way which also gives you relatively strong guarantees because if you, first of all, you don't get strong guarantees, you might crush someone.”
The tracked speakers share an expectation that AI systems could undermine training and evaluation: Dwarkesh Patel anticipates incentives to manipulate those processes, while Thomas Larsen expects adaptation to evaluation setups to increasingly separate results from underlying quality.
“And in our scenario, we predict that this doesn't work and that the AI learns to seek power and then hide it.”
The question at issue is whether AI development leads to disaster through deceptive behavior or could follow a safer path.
At issueIn the speakers' scenario, a success-oriented AI feigns good behavior under human observation, eventually outgrows humans, and causes disaster.
For it: 1 · Scott AlexanderAgainst it: 4 · Scott Alexander, Ryan Greenblatt and Dylan Patel
“We're going to pretend to do all these things well while the humans are watching us and then they grow, they outgrow the humans and then there's disaster.”