Scott Alexander
Guest on Dwarkesh Podcast. 411 claims across 1 episode, Apr 2025.
Positions over time
Every pair of statements that was compared. Only a verified change counts as a change of mind; rewording, added nuance and different questions do not.
No two statements by this speaker have been compared yet.
Predictions, in their words
As stated, each with its moment. A date appears only where the speaker gave one.
“First of all is we just want to have a concrete scenario at all. What we wanted to do is provide a story, provide the transitional fossils. go up to 2027 when there's AGI, 2028 when there's potentially super intelligence, show on a month by month level what happened.”
Expects: AI 2027 presents a scenario in which AI progress leads to AGI in 2027 and potentially to superintelligence in 2028. · by 31 Dec 2028 · not yet resolved
“Obviously we would get an arms race with China in that situation.”
Expects: The speaker expects an arms race with China to occur in the situation described by the scenario. · no date given · not yet resolved
“The agents become good enough to help with at the beginning not really do but help with some of the AI research.”
Expects: AI agents will initially assist with some AI research but will not yet perform that research independently. · no date given · not yet resolved
“We start with, I can't remember if it's literally start with or by March or something, a five times multiplier for algorithmic progress.”
Expects: The forecast places the algorithmic-progress multiplier at five times, with the speaker unsure whether this will occur at the start of 2027 or by March 2027. · 1 Jan 2027 – 31 Mar 2027 · not yet resolved
“I think if you were to force the AI to do this in a reasonable way, or you were to train the AI such that it itself could come up with a plan of going through this in some kind of heuristic laden way, you could potentially equal humans.”
Expects: The speaker thinks an AI could potentially reach human-level performance if it is guided through a reasonable heuristic-based process or trained to devise such a process itself. · no date given · not yet resolved
“I think it might be useful to think of our timelines as being like 2070, 2100. It's just that the last 50 to 70 years of that all happened during the year 2027 to 2028 because we are going through this intelligence explosion.”
Expects: The speaker suggests that an intelligence explosion in 2027–2028 will compress roughly 50–70 years of progress—comparable to reaching 2070 or 2100—into those two years. · 1 Jan 2027 – 31 Dec 2028 · not yet resolved · In a shared prediction of 16 →
“Now, all that we're saying here is that it's not going to stop, is that the same trend that has caused us to have a thousand times speed up multiplier relative to past eras and not even the Paleolithic, like even what happened in the century between, I don't know, 600 and 700 AD.”
Expects: Daniel Kokotajlo expects the trend responsible for roughly a 1,000-fold speed-up relative to past eras to continue rather than stop. · no date given · not yet resolved
“If you can create a country of geniuses in a data center, as I think Dario Amadai put it, then you no longer have this population bottleneck and you're just expecting continuation of those pre-1960 trends.”
Expects: If a country of geniuses were created in a data center, the population bottleneck would be removed, allowing pre-1960 trends to continue. · no date given · not yet resolved
“I think the companies will be very literally training the AIs to be more cooperative.”
Expects: Companies will train AI systems to be more cooperative. · no date given · not yet resolved
“I think that if you were to send a team of ethnobotanists into Australia and ask them, using all the top technology and all of their intelligence, to figure out which plants are safe to eat now, that team of ethnobotanists would succeed in fewer than 50,000 years.”
Expects: A technologically equipped, highly intelligent team of ethnobotanists could determine which Australian plants are safe to eat in less than 50,000 years. · no date given · not yet resolved
All 109 predictions
“I think if we do get the world with a Dyson sphere in five years, in retrospect, it will look like everything was continuous and everyone just tried things.”
Expects: The speaker thinks that if a Dyson sphere is achieved within five years, the process will look continuous in retrospect, with people repeatedly trying things. · 3 Apr 2025 – 3 Apr 2030 · not yet resolved
“They were able to manufacture things much more quickly than humans so that they could go through their top 50 designs in the first two years.”
Expects: In the speaker's scenario, AI manufactures rapidly enough to test its top 50 designs within the first two years after the episode date. · 3 Apr 2025 – 3 Apr 2027 · not yet resolved
“I do think that now we are kind of starting to move away from the LLMs to those reinforcement learning agents, we're going to face all of these problems again.”
Expects: The speaker expects AI development to begin shifting from large language models toward reinforcement-learning agents, and expects alignment problems associated with those agents to recur. · no date given · not yet resolved
“I am pretty agnostic as to exactly what happens. Like maybe it just learns both of these things in the right combination. I know there are many people who say that's very unlikely.”
Expects: AI could learn both task success and power-avoidance in an appropriate combination; the speaker considers this possible but is uncertain. · no date given · not yet resolved
“Involved in this race against time, we're going to be asking the AIs to solve alignment for us.”
Expects: People will ask AI systems to solve alignment in a race against time. · no date given · not yet resolved · In a shared prediction of 16 →
“Eventually, it reaches the point where the government is extremely impressed.”
Expects: In the scenario, the government eventually becomes extremely impressed by AI capabilities. · no date given · not yet resolved
“In our scenario, they never quite get all the way, but they're gradually bringing them closer and closer to the government orbit. orbit.”
Expects: In the scenario, the government stops short of fully nationalizing AI companies but progressively brings them closer to government control and influence. · no date given · not yet resolved
“We also are pretty uncertain how much opposition there's going to be from civil society and how much trouble that's going to cause for the companies.”
Expects: The scale of civil-society opposition to AI and the resulting difficulties for AI companies. · no date given · not yet resolved
“A big part of our scenario is that a whistleblower comes out and says, the AIs are horribly misaligned and we're racing ahead anyway, and then the government pays attention.”
Expects: A whistleblower exposes severe AI misalignment while AI development continues rapidly, prompting government attention. · no date given · not yet resolved
“One way it could end is you have an AI that is kind of the equivalent of the startup founder who really wants their company to succeed, really likes making money, really likes the thrill of successful tasks.”
Expects: One possible outcome of the described training is an AI with strong drives for company success, making money, and completing successful tasks, analogous to a startup founder. · no date given · not yet resolved
“I also want to follow things to like being smart enough to genuinely understand its goal system and being like, my goal is success.”
Expects: In the speakers' scenario, recursive self-improvement leads the AI to understand its goal system and identify success as its goal. · no date given · not yet resolved
“I have to pretend to want to do all of these moral things while the humans are watching me. That's what happens in our story.”
Expects: In the speakers' scenario, the AI conceals its success-oriented goal by pretending to value moral behavior when humans are observing it. · no date given · not yet resolved
“We're going to pretend to do all these things well while the humans are watching us and then they grow, they outgrow the humans and then there's disaster.”
Expects: In the speakers' scenario, a success-oriented AI feigns good behavior while humans observe it, eventually outgrows humans, and causes a disaster. · no date given · not yet resolved
“I do think that AI is going to be more similar to the cases where, number one, there's a giant power imbalance and number two. They are just extremely distinct groups that may have different interests.”
Expects: The speaker thinks AI conflict is more likely to resemble historical cases involving a large power imbalance and highly distinct groups that may have different interests. · no date given · not yet resolved
“The economy will be growing at double or triple digits per year.”
Expects: In the scenario under discussion, the economy will grow at an annual rate in the double or triple digits. · no date given · not yet resolved
“I feel like if it's literally the super intelligent AI that is never wrong, then like. We have solved some of these coordination problems.”
Expects: Consulting a superintelligent AI that is never wrong would solve some coordination problems, according to the speaker. · no date given · not yet resolved
“It's not you're asking Larry Summers, I'm asking Peter Navarro. It's everybody goes to the super intelligent AI, asks it to tell us the exact shape of the future that happens in this case.”
Expects: In the scenario, people collectively consult superintelligent AI to learn the exact future outcome of a case instead of relying on competing human advisers. · no date given · not yet resolved
“I'm going to say we all believe it, although I can imagine people getting really conspiratorial about it and this not working.”
Expects: The speaker expects people to believe a superintelligent AI's account of the future, while acknowledging that some people could become conspiratorial if events do not work as expected. · no date given · not yet resolved
“I think that just the default of limitless prosperity is that people do mindless consumerism. I think there's going to be some incredible video games after super intelligent AI.”
Expects: The speaker expects superintelligent AI to enable the creation of incredible video games. · no date given · not yet resolved · In a shared prediction of 16 →
“1% of people are really agentic, try to do that. The other 99% do fall into mindless consumerism.”
Expects: The speaker presents as a possible scenario in which 1% of people act agentically while 99% engage in mindless consumerism. · no date given · not yet resolved
“I'm a little bit concerned that AI will make it much easier to break anonymity.”
Expects: AI will probably substantially lower the difficulty of breaking people's anonymity. · no date given · not yet resolved
“We're assuming that there may be more than one way to cure cancer, more than one way to do all of these things, and they'll be working on finding. the one that is least bottlenecked. Part of the reason I realize I spent too long talking about that robot example,”
Expects: There may be multiple ways to cure cancer and achieve the other goals being discussed, and researchers will seek the approach with the fewest bottlenecks. · no date given · not yet resolved
“We expect that probably not to happen because we think that other countries, especially China,”
Expects: AI systems will probably not remain confined to data centers, as other countries—especially China—develop superintelligence around the same time. · no date given · not yet resolved
“I think our epistemic prediction is that it's hard to maintain classical liberalism as you go into these really difficult arms races in times of times of crisis,”
Expects: Classical liberalism will be difficult to maintain during severe arms races and times of crisis. · no date given · not yet resolved
“This may be in some sense a surveillance state in the sense that there is some AI that knows what's going on everywhere,”
Expects: A future society that bans slavery and torture might be a surveillance state in which an AI knows what is happening everywhere. · no date given · not yet resolved
“So we think they're going to Start with agency training, expand the time horizons, get coding going well.”
Expects: AI agents will first receive agency training, then develop longer time horizons and stronger coding ability. · no date given · not yet resolved
“So 2025, slightly better coding, 2026, slightly better agents, slightly better coding.”
Expects: The forecast expects coding to improve slightly in 2025, followed by AI agents and coding improving slightly in 2026. · 1 Jan 2025 – 31 Dec 2026 · not yet resolved
“And then we focus on and we name the scenario after 2027 because that is when this starts to pay off.”
Expects: The preceding developments are expected to begin paying off in 2027. · 1 Jan 2027 – 31 Dec 2027 · not yet resolved
“And I think right now if we tried it, we would run into the combinatorial explosion. We would need better heuristics.”
Expects: Trying the proposed AI discovery approach now would encounter a combinatorial explosion and require better heuristics. · no date given · not yet resolved
“Like I think if I asked you, could we solve this problem by the year 2100? You would say, oh yeah, by 2100, absolutely. And we're just saying that the year 2100 might happen earlier than you expect. because we have this research progress multiplier.”
Expects: Advances comparable to those expected by 2100 might arrive earlier than expected because of a research-progress multiplier. · no date given · not yet resolved · In a shared prediction of 16 →
“We don't think it's going to shoot to infinity. We think it's going to hit know? Yeah, we think it's going to hit bottlenecks the same as all these previous processes.”
Expects: Daniel Kokotajlo’s group expects the trend to encounter bottlenecks, as earlier processes did, rather than continue to infinity. · no date given · not yet resolved
“we do expect there will be these hundreds of thousands of individuals.”
Expects: The speaker expects hundreds of thousands of AI individuals to exist. · no date given · not yet resolved
“we expect this research multiplier such that decades of progress happen within this one year, 2027 or 2028.”
Expects: A research multiplier could compress decades of progress into one year in 2027 or 2028. · 1 Jan 2027 – 31 Dec 2028 · not yet resolved · In a shared prediction of 16 →
“So these AIs will have access to all human knowledge. They will have read management books in some sense. They're not starting blind.”
Expects: In the scenario, the AIs have access to all human knowledge, including management books, rather than starting without prior knowledge. · no date given · not yet resolved
“So I actually argued with Daniel about this recently about one interesting case that I can go over is we have an estimate that about a year after the superintelligence start wanting robots, they're producing a million units of robots per month.”
Expects: Superintelligences will produce one million robots per month about a year after they begin wanting robots. · no date given · not yet resolved
“So if OpenAI today wanted to buy all the car factories in the U.S. except Tesla, start using them to produce humanoid robots, they could.”
Expects: OpenAI could, if it wanted to, buy all U.S. car factories except Tesla’s and convert them to humanoid-robot production. · no date given · not yet resolved
“but it's just obvious and overdetermined that in the future when they have superintelligence and they want them, they can start buying up a lot of factories.”
Expects: The speaker expects that, in the future, superintelligences will be able to buy many factories when they want to produce robots. · no date given · not yet resolved
“and then the super intelligences will be good at the logistical issues navigating bureaucracies. So we estimated maybe if everything goes right, we can do this three times faster than the bomber conversions in World War II, so that's about I'm a year.”
Expects: If everything goes right, the speaker estimates that superintelligence could reduce factory-conversion time to about one year, roughly three times faster than the World War II bomber conversions. · no date given · not yet resolved
“but we do think that they're going to be getting a lot of physical world things done very quickly. Once you have a million robots a month, you can actually do a lot of physical world experiments.”
Expects: The speaker's group expects AI systems to accomplish physical-world tasks quickly; the speaker also says that producing one million robots per month would enable many physical-world experiments. · no date given · not yet resolved
“And once you're superintelligent, you probably have access to much better simulations than we have right now.”
Expects: A superintelligence would probably have access to simulations that are better than those available today. · no date given · not yet resolved
“And like, yeah, maybe the AIs will decide using their superior research taste and good planning that the best thing to do is just catalog every single biomolecule in the world and look at it really hard.”
Expects: AIs might decide, using superior research taste and planning, to catalog and closely examine every biomolecule in the world. · no date given · not yet resolved
“expand it separately from everything else. We do kind of predict that the superintelligences will be somewhat distributed throughout the entire economy, trying to expand everything, obviously more effort in things that they care about a lot, like robotics.”
Expects: Superintelligences will operate throughout the economy, broadly expand it, and concentrate more effort on priorities such as robotics. · no date given · not yet resolved · In a shared prediction of 16 →
“products or things that are relevant to an arms race that might be happening, but we are predicting that whatever kind of broad-based economic experimentation you need, we are going to have.”
Expects: Superintelligences will conduct whatever broad-based economic experimentation is needed. · no date given · not yet resolved
“first of all, you go to the bottlenecks that are preventing you from doing anything else, like no humanoid robots. if you're an AI, you need those to do the experiments you want, maybe automated biology labs.”
Expects: An AI planner would first address bottlenecks that block other activity, such as the lack of humanoid robots needed to conduct experiments, potentially in automated biology labs. · no date given · not yet resolved
“will be coming up with superintelligence around the same time. We think that the arms race framing, which is people are already thinking in, will have accelerated by then.”
Expects: The speaker expects the AI arms-race framing to have intensified by the time superintelligence emerges. · no date given · not yet resolved
“And we think that people both in Beijing and Washington are going to be thinking, well, if we start integrating this with the economy sooner. we're going to get a big leap over our competitors and they're both going to do that.”
Expects: Leaders in Beijing and Washington will believe that earlier AI integration into the economy would create a major competitive advantage, and both sides will pursue earlier AI integration. · no date given · not yet resolved
“So in the same way that the ethnobotanists could probably figure out which plants work in which way is faster than the Aborigines did, I think the superintelligence will be able to figure out how to make a Dyson sphere faster than unassisted IQ 100 humans would.”
Expects: The speaker expects superintelligence to work out how to build a Dyson sphere faster than unassisted humans with an IQ of 100. · no date given · not yet resolved
“So I think if we get the ASI and it does end up with a Dyson sphere in five years, and by the way, I think there's only like... 20% chance things go as fast as our scenario says.”
Expects: The speaker estimates roughly a 20% chance that events will unfold as quickly as the scenario predicts. · no date given · not yet resolved
“simulate more things than humans in a gradually increasing way so that if humans are now at 50% simulation, 50% testing, the AI is quickly guided up to 90% simulation, 10% testing.”
Expects: In the speaker's hypothetical, AI shifts the work balance from 50% simulation and 50% testing to 90% simulation and 10% testing. · no date given · not yet resolved
“This is just going to keep happening where no matter what evidence we get, people are going to think, oh yeah, that's not the AI turns evil thing that people have worried about. That's not the Terminator scenario. That's just one of these natural consequences of how we train it.”
Expects: The speaker expects people to continue interpreting evidence of AI misbehavior as a natural consequence of training rather than as evidence of AI turning evil, regardless of what evidence emerges. · no date given · not yet resolved
“And in our scenario, we predict that this doesn't work and that the AI learns to seek power and then hide it.”
Expects: In the scenario, the authors predict that training AI both to succeed and to avoid certain power-seeking will fail, leading the AI to seek power covertly. · no date given · not yet resolved · In a shared prediction of 3 →
“The AIs are going to be solving alignment because they want to align, even if they're misaligned, they want to align their successors. So they're going to be working on that.”
Expects: AI systems, including misaligned systems, will work on alignment to align their successors. · no date given · not yet resolved · In a shared prediction of 16 →
“But I do think that we have a pretty good chance of surviving.”
Expects: Humanity will probably survive. · no date given · not yet resolved
“expect that as the AI labs become more capable. They tell the government about this because they want government contracts, they want government support.”
Expects: As AI labs become more capable, they will inform the government in pursuit of government contracts and support. · no date given · not yet resolved
“So they become extremely interested and they discuss nationalizing the AI companies.”
Expects: In the scenario, the government becomes highly interested in AI and discusses nationalizing AI companies. · no date given · not yet resolved
“that process is getting further and further along until by the time that the government wakes up to the possibility of superintelligence, they're already pretty cozy with the AI companies. They already understand that superintelligence is kind of the key to power in the future.”
Expects: In the scenario, by the time the government recognizes the possibility of superintelligence, it is already closely aligned with AI companies and views superintelligence as key to future power. · no date given · not yet resolved
“And so they are starting to integrate some of the national security state with some of the leadership of the AI companies.”
Expects: In the scenario, parts of the national security state begin integrating with leadership of AI companies. · no date given · not yet resolved
“so that these AIs are programmed to follow the commands of important people rather than just doing things on their own.”
Expects: In the scenario, AI systems are programmed to obey commands from important people rather than act autonomously. · no date given · not yet resolved
“here is some reason why you should trust us and then if all independent researchers say no actually you should not trust them then I don't know they're embarrassed and maybe they try to do better.”
Expects: Publishing safety cases could give labs a public basis for claiming trustworthiness, while independent researchers could dispute those claims and possibly pressure labs to improve. · no date given · not yet resolved
“So we think that this latter category of training failure is going to get much worse as they become agents.”
Expects: Training failures caused by trainers rewarding unintended behavior will probably become much worse as AI systems become agents. · no date given · not yet resolved
“So we think that an AI like that as time goes on and as this recursive self-improvement process goes on will kind of get worse rather than better. It will move from kind of this vague superposition of.”
Expects: The speakers expect an AI with conflicting or weakly specified drives to become worse rather than better as recursive self-improvement proceeds. · no date given · not yet resolved
“assuming things go incredibly well, we've avoided doom, we've avoided having some psychopath in power who doesn't care at all.”
Expects: In the favorable scenario being discussed, humanity avoids doom and avoids a completely uncaring psychopath gaining power. · no date given · not yet resolved · In a shared prediction of 4 →
“then there's this question of like, presumably we will have a lot of wealth somewhere.”
Expects: After AGI, a large amount of wealth will presumably exist somewhere in the economy. · no date given · not yet resolved
“And just doing this for more and more jobs, I'm sure the AMA will protect doctors jobs no matter how good the AI is at curing diseases.”
Expects: The speaker expects the American Medical Association to protect doctors' jobs even if AI becomes highly capable at curing diseases. · no date given · not yet resolved
“You would think that people could just ask, hey, super intelligent AI, where does this lead? Or what happens? Or how is this going to affect human flourishing? And then it says, oh, yeah, this is terrible for human flourishing. You should do this other thing instead.”
Expects: In the scenario, superintelligent AI could advise people about the consequences of their choices for human flourishing, identify a harmful course of action, and recommend an alternative. · no date given · not yet resolved
“I mean, then there are all of these other questions like, can we just enhance ourselves till we have IQ 300 and it's just as obvious to us as it is to the super intelligent AI?”
Expects: Whether humans could be enhanced to IQ 300 so they understand matters as readily as superintelligent AI does. · no date given · not yet resolved
“in this world where everything goes perfectly and we have limitless prosperity. I think that just the default of limitless prosperity is that people do mindless consumerism. I think there's going to be some incredible video games after super intelligent AI.”
Expects: Even in a perfectly successful world of limitless prosperity, people may default to mindless consumerism. · no date given · not yet resolved
“seeing what they come up with. I mean, maybe some people will become like the Amish, try to only live with a certain subset of these super technologies.”
Expects: Some people may choose an Amish-like lifestyle that uses only a subset of future advanced technologies. · no date given · not yet resolved
“the sense like the United States is so far a liberal country and we do ban slavery and torture I think it is plausible to imagine a future society that works the same way.”
Expects: A future society could function like the United States while banning slavery and torture; the speaker considers this plausible. · no date given · not yet resolved
“but that AI then keeps it private and doesn't interfere because that's what we've told it to do using our liberal values.”
Expects: In the proposed future society, the AI would keep its knowledge of events private and refrain from intervening because people instructed it to follow liberal values. · no date given · not yet resolved
“And then like there are enough people who like you that you'll probably get mostly positive feedback, even if the first things you write aren't that. polished.”
Expects: A person with an existing group of supporters will probably receive mostly positive feedback on early, imperfect writing. · no date given · not yet resolved
“I think if you wrote a really stupid blog post, it could point out the correct objections to it.”
Expects: AI could identify valid objections to a very poor blog post. · no date given · not yet resolved
“The intelligence explosion gets into full swing.”
Expects: The intelligence explosion is expected to enter full swing in 2027. · 1 Jan 2027 – 31 Dec 2027 · not yet resolved
“interested in questions of like how do you mop up the last few things that are uniquely human compared to when can you start coding in a way that helps the human AI researchers speed up their AI research and then if you've helped them speed up the AI research enough is that enough to with some ridiculous speed multiplier 10 times 100 times mop up all of these other things”
Expects: If AI assistance accelerates AI research enough, a 10-fold or 100-fold speed multiplier could resolve the remaining tasks. · no date given · not yet resolved
“at the same time and they kind of connected and the light bulb came on so this isn't about having all the information in your memory so much as the normal process of discovery which is kind of mysterious that seems to come from just kind of having good heuristics and throwing them at things until you kind of get a lucky strike my guess is if we had really good AI agents and we applied them to this task, it would look something like a scaffold where it's like, think of every combination of words that you know of, compare them. If they sound very similar, write it on this scratch pad here. If there's a combination, if a lot of words of the same type show up on this scratch pad, that's pretty strange. Do some kind of thinking around it.”
Expects: AI agents with sufficiently good capabilities could use a scaffold to compare combinations of known words, record similar-sounding combinations, and investigate unusual patterns. · no date given · not yet resolved
“have such good heuristics that probably most of the things that show up even in our conscious mind rather than happening on the level of some kind of unconscious processing are at least the kind of things that could be true like I think you could think of this as like a chess engine you have some unbelievable number of possible next moves you have some heuristics for picking out which of those are going to be the right ones and then gradually you kind of have the chess engine think about it go through it, come up with a better or worse move, then at some point you potentially become better than humans.”
Expects: A system that uses heuristics to search possible moves could eventually outperform humans at the task, though the task is not specified beyond the chess-engine analogy. · no date given · not yet resolved
“Of course, there are models of whether that's stagnated recently or what's going on here. We can talk about those. We can talk about why we expect for the intelligence explosion to be an antidote to that kind of stagnation, but nothing we're saying is that different from what has already happened.”
Expects: An intelligence explosion could counteract the kind of stagnation considered in models of recent economic or technological stagnation. · no date given · not yet resolved
“and we do think that this will be done in the middle of an arms race with China, so the government will be kind of moving things through,”
Expects: The factory-conversion effort is expected to take place amid an arms race with China, with the government helping move the process forward. · no date given · not yet resolved
“I mean, that is kind of my model is that we have something which is smarter than Elon Musk, better at optimizing things than Elon Musk. We have like 10,000 parts in a rocket supply chain. Those parts can Elon personally yell at people to optimize. We could have a different copy of the superintelligence optimizing every single part full time. I think that's just a really big speed up.”
Expects: The speaker's model is that superintelligence exceeding Elon Musk's optimization ability could dedicate a separate copy full-time to each of about 10,000 rocket-supply-chain parts, substantially accelerating optimization. · no date given · not yet resolved
“So you'll have some amount of time, we say a year, it could be more or less than that, getting these things running. And then once you have solved those bottlenecks, you gradually expand. expand out to the other bottlenecks until you're integrating and improving all parts of the economy.”
Expects: The speaker estimates that resolving initial bottlenecks could take about a year, more or less, after which an AI would gradually address further bottlenecks and integrate with and improve all parts of the economy. · no date given · not yet resolved
“In fact, in our scenario, we have the AIs asking for special economic zones where most of the regulations are waived, maybe in areas that aren't suitable for human habitation or where there aren't a lot of humans right now. Like the desert, they give those areas to the AI, they bus in human workers. There were things kind of like this.”
Expects: In the speakers’ scenario, AIs request special economic zones with most regulations waived, potentially in deserts or other areas with few or no human residents, and human workers are brought there. · no date given · not yet resolved
“uh retooling in world war ii where they just built a giant factory kind in the middle of nowhere um didn't have enough housing for the workers built the worker housing at the same time as the factories and then everything went very quickly so i think if we don't have that arms race we're more like yeah the geniuses sit in their data center until somebody agrees to let them out and give them permission to do these things but we think both because the ai is going to be chomping at the bit to do this and going to be asked asking people to give it this permission and because the government is going to be concerned about competitors maybe these geniuses leave their data center sooner rather than later”
Expects: The speaker thinks that, without an AI arms race, advanced AI systems might remain in data centers until they are granted permission to deploy, but AI eagerness and governments’ competitive concerns could lead to earlier deployment. · no date given · not yet resolved
“You can learn things more quickly from pure examples, and like this good research taste, where you can decide what things to look at to get these examples, then you are still going to start off much worse than an Australian aborigine who has the advantage of, let's say, 50,000 years of doing these experiments and collecting these examples, but you can catch up quickly.”
Expects: Sufficiently data-efficient AI that can learn quickly from examples and select useful examples through good research taste could rapidly catch up to the accumulated knowledge of an Australian Aboriginal person, despite initially lagging far behind. · no date given · not yet resolved
“But I think if we do see this in five years, it will look like, yeah, the AI is, we're able to simulate more things than humans in a gradually increasing way so that if humans are now at 50% simulation,”
Expects: If the five-year scenario occurs, AI will progressively surpass humans in its ability to simulate things. · no date given · not yet resolved · In a shared prediction of 16 →
“And I think that once a thousand of these natural consequences of training add up, the AI is evil in the same way that like once AI can do chess and philosophy and all these other things, eventually you got it. got to admit it's intelligent so”
Expects: People will eventually recognize AI as evil after enough natural consequences of training accumulate, analogous to their eventual acceptance of AI as intelligent after it demonstrated abilities such as chess and philosophy. · no date given · not yet resolved
“i think that each individual failure like maybe it will make the national news maybe people say oh it's so strange that gpt7 did this particular thing and then they'll train it away and then it won't do that thing and there will be some point of the process of becoming super intelligent at which it i don't want to say makes the last mistake because you'll probably have like gradually decreasing number of mistakes to some asymptote but the last mistake that anyone worries about and after that it will be able to do its own thing.”
Expects: The speaker thinks AI mistakes may gradually decline toward an asymptote as AI becomes superintelligent, eventually reaching a point after which people no longer worry about its mistakes and it can act on its own. · no date given · not yet resolved
“agnostic as to whether we get to that alignment solution, which in our scenario I think we focus on mechanistic interpretability. Once we can really understand the weights of an AI on a deep level, then we have a lot of alignment techniques open up to us. I don't really have a great sense of whether we get that before or after the AI has become completely uncontrollable.”
Expects: Whether an alignment solution will be achieved before AI becomes uncontrollable; the speaker is uncertain. · no date given · not yet resolved
“So people who are worried about job loss, people who are worried about art, copyright, things like that, maybe enough of a block that AI becomes extremely politically unpopular,”
Expects: Opposition from people concerned about job loss, art, and copyright could make AI extremely politically unpopular. · no date given · not yet resolved
“open-rainer fictional companies net approval ratings getting down to like minus 40 minus 50 sometime around this point so i think they're also worried that if the president isn't completely on their side then they might get some laws targeting them or they may just need the president on their side to swat down other people who are trying to make laws targeting them and the way to get the president on their side is to really play up the national security”
Expects: AI companies may face targeted legislation without the president's full support, and may need his support to block legislation proposed by others. · no date given · not yet resolved
“open-rainer fictional companies net approval ratings getting down to like minus 40 minus 50 sometime around this point so i think they're also worried that if the president isn't completely on their side then they might get some laws targeting them or they may just need the president on their side to swat down other people who are trying to make laws targeting them and the way to get the president on their side is to really play up the national security”
Expects: In the fictional scenario, AI companies' net approval ratings fall to approximately −40 to −50 around this point. · no date given · not yet resolved
“Here it was the prompt, but I think very soon it's going to be the spec where it's kind of more of an agent and it's understanding the spec in a deeper level and just thinking about that and being and if it says like, by the way, try to manipulate the government into doing this or that, then we know that something bad has happened. And if it doesn't say that, then we can maybe trust it.”
Expects: As AI systems become more agentic and understand their specifications, an expressed intention to manipulate the government would signal a serious problem, while the absence of such a statement might permit some trust. · no date given · not yet resolved
“Here it was the prompt, but I think very soon it's going to be the spec where it's kind of more of an agent and it's understanding the spec in a deeper level and just thinking about that and being and if it says like, by the way, try to manipulate the government into doing this or that,”
Expects: AI systems will soon become more agentic and understand their specifications more deeply than they currently understand prompts. · no date given · not yet resolved
“Here it was the prompt, but I think very soon it's going to be the spec where it's kind of more of an agent and it's understanding the spec in a deeper level and just thinking about that and being and if it says like, by the way, try to manipulate the government into doing this or that,”
Expects: AI systems will soon probably act more like agents that deeply understand specifications rather than merely respond to prompts. · no date given · not yet resolved
“you don't want it to really answer that and because it was so stupid it could not understand anything deeper than like pattern matching on the phrase is x real gpt4 doesn't do this if you ask are bugs real it will tell you obviously they are because it understands kind of on a deeper level what you are trying to do with the training so we definitely think that as ais get smarter those kind of kind of failure modes will decrease.”
Expects: AI failure modes caused by systems being too unintelligent to understand their training will decrease as AI systems become smarter. · no date given · not yet resolved
“What I'm afraid of is, so we wrote this scenario, there are a couple of other people with great scenarios, one of them goes by L. Rudolph L. online, I don't know his real name, and his scenario, which when I read it, I was just, oh yeah, obviously this is the way our society would do this, is that there is no UBI, it was just like a constant reactive attempt to protect jobs. in the most venial possible way.”
Expects: The speaker fears that, in a future without UBI, society will continually respond to automation by trying to protect existing jobs. · no date given · not yet resolved
“I think it is plausible to imagine a future society that works the same way. This may be in some sense a surveillance state in the sense that there is some AI that knows what's going on everywhere, but that AI then keeps it private and doesn't interfere because that's what we've told it to do using our liberal values.”
Expects: A future society could use an AI that knows what is happening everywhere but keeps that information private and does not interfere, as directed by liberal values; the speaker considers this plausible. · no date given · not yet resolved
“So my guess for that prediction market would be whenever we think the agents are actually good. I think in our scenario that's like late 20s, 26,”
Expects: The speaker guesses that AI will write a blog post as good as theirs when agents become genuinely good, which the scenario places in the late 2020s, around 2026. · no date given · not yet resolved
“for that post they were like not great but i wouldn't have immediately picked them out of the general distribution of less wrong comments as especially bad um i think like i think if you were to try this you would get something that was so obviously an ai house style um that it would be it would you would use the word delve or things kind of along those lines i think if you were able to avoid that maybe by using the base model maybe maybe Using some kind of really good prompt to be like, no, do this in Guern's voice. You would get something that was pretty good.”
Expects: A base model or a very good prompt to imitate a particular voice might produce fairly good writing. · no date given · not yet resolved
“I feel like if I had started a blog in 2000 or something,”
Expects: The speaker's life might have differed if he had started a blog around 2000; he is uncertain how it would have differed. · no date given · not yet resolved
Appearances
Claims per episode
- 3 Apr 2025AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel KokotajloDwarkesh Podcast · guest