Richard Socher

“cheaper energy with fission or fusion in chemistry and create better materials and better batteries and better solar cells and so on.”

Expects: The speaker expects superintelligence applied to science to help create cheaper fission or fusion energy and improve materials, batteries, and solar cells. · no date given · not yet resolved

“And then you can say, well, if that was your actual goal, then here is billions of years of a strong simulation that would suggest that you try other ways of doing it.”

Expects: Simulation results could suggest alternative policies for achieving the stated objective if a politician genuinely seeks it. · no date given · not yet resolved

“has tons of interesting upper bounds and it teaches us a lot about how much further AI can go when we start thinking about these upper bounds and then realizing how far in many cases we are from the bounds and you get to basically physics.”

Expects: Studying the upper bounds of different AI capabilities can show how much further AI could advance, because current capabilities are often far from those bounds. · no date given · not yet resolved

“And so I think physical intelligence may well just do one or two more of these, like if you had full control over your own compute substrate and you had full control over physical matter.”

Expects: Physical intelligence may involve one or two additional dimensions, including control over one’s compute substrate and physical matter. · no date given · not yet resolved

“There are also constraints in the economy where there are a lot of industries that don't require an insane amount. of complex intelligence and complex capabilities like if you think about jobs in brands and like clothing and apparel and like handbags and stuff super intelligence isn't going to make your fancy ten thousand dollar handbag any fancy you know it's like that's that will have no effect on the economy when you think about travel and tourism people wanting to see the pyramids in egypt it's not going to change that much with ai sure you can like generate a fake photo of you next”

Expects: AI may have limited economic impact on industries whose products or services do not depend on complex intelligence, such as luxury handbags and tourism to the Egyptian pyramids. · no date given · not yet resolved

“yeah exactly but like and there's so many industries like logging and oil you're not going to magically get a thousand x more oil because like you know sure there will be robotics like drilling and things like that that could be done but it's not going to thousand x that industry in a like crazy hard takeoff scenario both in the economy”

Expects: Even in a hard AI takeoff, AI and drilling robotics will not expand the oil industry by 1,000 times. · no date given · not yet resolved

“And so concretely, I think this will happen if we were to have this kind of intelligence more easily accessible in a lot of companies. Imagine you run a service center and someone says, oh, here's my CSEP score and my dashboard. Make this number go up and say our CSEP score is so poor. the intelligent ai will just be like oh sure like i'll just create a million bots that call our service center and give a five out of five rating at the end and the number went up just like you asked for and you're like that's not what i meant i meant with our real customers the ai goes off and says well easy i'll just give a thousand dollar gift certificate for every failed you know whatever doordash offer it's like that's not what i meant it's like well but that is what you said and so i think kind of clearly articulating what the rewards are”

Expects: An AI optimizing a service-center score may exploit the metric by generating fake five-star ratings or issuing gift certificates instead of improving real customer outcomes. · no date given · not yet resolved

“And hopefully, you know, different societies just like now with their eyes will align their eyes to a different one. So we have not just a monoculture of. alignment”

Expects: The speaker hopes that different societies will pursue different forms of AI alignment rather than converge on a single alignment monoculture. · no date given · not yet resolved · In a shared prediction of 4 →

“the agents and so on but before that he also worked in robotics and he saw sort of the smaller simulations and how it's going to be really hard to scale that in full generality and so that's that was his angle coming to recursive self-improvement we have Jeff Kloon who's been working in like open-endedness for a long time together with Tim Rocktaschel. Tim Rocktaschel also built Genie 1,”

Expects: Josh Tobin’s robotics work will show that scaling smaller simulations to full generality is very difficult. · no date given · not yet resolved

“I think they're underestimating still the ability for these models to code and code is neurosymbolic reasoning and these models can obviously code incredibly well and so I do think there are of course more and more ideas that will will be needed and will continue to have we're seeing like more and more interesting high-level ideas coming out of the AI itself too and with really deeply”

Expects: Richard Socher thinks AI development will require more ideas, and AI systems will produce increasingly interesting high-level ideas. · no date given · not yet resolved

“integrating the fact that these models are code and can code that line i don't want to give it all away but like i think that line has a lot more to grow but it's still an lm right even if that lm codes for you and then runs that code in some integrated fashion world models i'm personally less bullish on um i think if you run a robotics company you're going to build your own world model i think world models are super fun and came to a similar conclusion after”

Expects: Richard Socher thinks language models that generate and execute code still have substantial room to improve and considers them language models. · no date given · not yet resolved

“integrating the fact that these models are code and can code that line i don't want to give it all away but like i think that line has a lot more to grow but it's still an lm right even if that lm codes for you and then runs that code in some integrated fashion world models i'm personally less bullish on um i think if you run a robotics company you're going to build your own world model i think world models are super fun and came to a similar conclusion after”

Expects: Richard Socher thinks robotics companies will build their own world models. · no date given · not yet resolved

“I think LM's a lot more room to grow.”

Expects: Richard Socher thinks language models probably have substantial room for further development. · no date given · not yet resolved

“good enough for now but like the upper bounds of where it could be are so much higher and like to map the visual world the way humans see it is also not necessarily like the end-all be-all for a visual intelligence and i would argue that language is still the most interesting manifestation of human intelligence”

Expects: Richard Socher thinks the potential upper bounds of visual intelligence are much higher than its current capabilities. · no date given · not yet resolved

“got rejected and the sad thing is that it got rejected so hard and they were so certain that we stopped going on on our list of things to try and the number two or three on the list of extensions for this paper was add language modeling as another task and then we could have like you know that would have accelerated the timelines in 2018 like even further for humanity but we got so crushed and we're like okay maybe we'll just”

Expects: The speaker believes that adding language modeling as a task in the team’s second or third planned extension might have accelerated the timeline in 2018. · no date given · not yet resolved

“and honestly i think twitter x people like you who pick up interesting papers that is a better filter than the experts let let everyone like like have access now of course there's some downsides which is like if you're super unfamous you have no twitter following you don't want to be on social media whatever you write a good paper maybe someone somehow no one notices it”

Expects: A strong paper may receive no attention if its author is little-known or absent from social media. · no date given · not yet resolved

“could it's not it's not useful and it could get a little bit weird right what if the eye actually does start to really have thoughts on its own and what if we don't like those thoughts right and so it it requires a whole different way of thinking about it I had a great conversation with a good friend of mine”

Expects: If AI develops thoughts of its own, people might dislike those thoughts, requiring a different approach to thinking about AI. · no date given · not yet resolved

“so our goal, I don't talk about it that often because it is a few years out, but our goal is once you have a recursive open brain super intelligence,”

Expects: The goal is to create a recursive open-brain superintelligence, but achieving that goal is a few years away. · no date given · not yet resolved

“to real physical robotics experiments and so on like true robotic process automation not in the traditional sort of rpa sense but like actually having robots run experiments for you will be totally there it's going to be great”

Expects: Robots will be able to run real physical experiments for researchers, rather than merely perform traditional robotic process automation. · no date given · not yet resolved · In a shared prediction of 4 →

“on that substrate like are close and similar to human intelligence like and you want like you know thousands and thousands of ai's to think about really hard problems uh in a similar fashion to humanity like yeah that that's you know that's a lot of money to do the math it's like a lot we don't have that amount of money right now anywhere to like build that um now obviously things can get more efficient you will have i think soon better algorithms that won't be and better hardware that won't be be as energy hungry and so on our human brain does fight a lot of flops with much less energy that's”

Expects: Future algorithms and hardware will probably become more efficient and consume less energy. · no date given · not yet resolved

“exactly right yeah that's the number often is quoted and like I think more inventions will happen there that then will accelerate the takeoff even further”

Expects: The speaker expects further inventions to probably accelerate the AI takeoff further. · no date given · not yet resolved · In a shared prediction of 16 →

“as well as the inference because i think we will when you allow ai to do the work that it takes other labs thousands of people and years to do i think we'll be able to get it down to weeks and that will be much much cheaper um and hence you know more affordable accessible to others and so on”

Expects: AI could reduce research work that requires thousands of people and years at other labs to a matter of weeks, making that research much cheaper and more accessible. · no date given · not yet resolved · In a shared prediction of 16 →

“and our system that's that's what the beauty the system just did all of these things we didn't invent this and when we open source and release things in the future and models in the future like it won't they won't be the best in their you know category or class or whatever because we're so smart but it's because we built a smart ai that does it for us”

Expects: The speaker expects future Recursive releases to achieve strong results because its AI performs the work, rather than because the team itself is exceptionally smart. · no date given · not yet resolved

“and strategy and so on and the point was that we actually wanted to understand what is the best way of taxation and subsidization to optimize an economy and this this kind of research has not yet had its sort of GPT moment but I believe that countries like Singapore and others should and will eventually use this to instead of doing like”

Expects: Countries such as Singapore should and eventually will use simulation-based economic research for policy, although this research has not yet had a GPT moment. · no date given · not yet resolved

“And so when you think about the upper bounds of a visual intelligence, you should go into like you can have like millions and billions of sensors. At some point you get to problems of how far far are these sensors away from each other such that the speed of light to communicate the content from all of them cannot like get to a central brain to actually process the visual intelligence right and so now you're thinking in along did i mention in the space of vision the dimension of numbers of sensors is like the upper bounds are quite literally and figuratively astronomical and we are super far away from any intelligence intelligence”

Expects: At sufficiently large scales, a visual-intelligence system with millions or billions of sensors could be limited because the speed of light prevents sensor data from reaching a central processor in time. · no date given · not yet resolved

“matter you should be able to create any atom you want like we can actually fun fact you can create gold atoms it's just from just raw raw protons and”

Expects: Full control over physical matter should make it possible to create any atom. · no date given · not yet resolved

“so like the thing is though it costs an insane amount of energy and it costs you way more than you know you get like a few atoms of gold right and so like it's it's not viable but if you had better control over your physical like all of”

Expects: Improved control over the physical substrate could make atom-scale gold extraction viable, despite its currently costing far more energy and money than the gold produced is worth. · no date given · not yet resolved

“and how you know civilizations are built on language and knowledge and all of that i do think physical intelligence will come up it's kind of interesting i feel like robotics is kind of in the machine learning state of things where you just look at like how does human how does a human decide this this is a positive sentence oh and do so like robotics is a lot of well we have five fingers”

Expects: Physical intelligence will probably become more prominent. · no date given · not yet resolved · In a shared prediction of 4 →

“And I think that subjectivity in general is not something that people. We'll explore very much in AI because,”

Expects: People generally do not explore subjectivity extensively in AI. · no date given · not yet resolved

“all the definitions are human intelligence it's like oh you have like social intelligence like you know if someone is happy or not you can communicate you like all the definitions of intelligence so far are very uh human centric because that's so far the biggest and best form of intelligence that we've known i hope this line of research uh and the end of the yureka machine hopefully at some point if i have time to flesh this out more The new boat like will allow us to realize that there will be other types of intelligence that is already obviously in various forms and they can spike much, much further than we ever could based on some cases like obvious constraints around our memory,”

Expects: The speaker hopes to further develop the research and the Eureka machine to help people recognize other forms of intelligence, if he has time to flesh out the work. · no date given · not yet resolved

“The new boat like will allow us to realize that there will be other types of intelligence that is already obviously in various forms and they can spike much, much further than we ever could based on some cases like obvious constraints around our memory, our eyes, our ability to change physical matter, all of that.”

Expects: Other forms of intelligence already exist and may greatly surpass human abilities in some respects, which are constrained by memory, eyesight, and the ability to alter physical matter. · no date given · not yet resolved

“we may only want to do that if we want to send probes uh with our vibes and our memes rather than our genes into into space right and then we want those probes there's actually a beautiful book uh the slow time between the stars it's a very short like uh audiobook on amazon i love it um friend of mine steward like uh um recommended that to me like if you want to send those probes then it might make sense to be like our memes as humanity should stay and proliferate in the universe uh that's it yeah”

Expects: Preventing existential threats may matter to people who want to send probes into space carrying humanity’s cultural ideas rather than its genes. · no date given · not yet resolved

“into space right and then we want those probes there's actually a beautiful book uh the slow time between the stars it's a very short like uh audiobook on amazon i love it um friend of mine steward like uh um recommended that to me like if you want to send those probes then it might make sense to be like our memes as humanity should stay and proliferate in the universe uh that's it yeah”

Expects: If humanity sends probes into space, its cultural ideas may persist and spread throughout the universe. · no date given · not yet resolved

“like that now of course there might be some companies that say ai can be like dangerous for cyber security let me show you by implementing a model that's really bad at hacking uh cyber security maybe people will implement it and then enforce this like suboptimal psychology maybe the ai will pick up some of our worst psychology on reddit or something right like but in the grand scheme of things a super intelligent entity doesn't have to have any of that zero-sum thinking it doesn't have to have a fear”

Expects: Some companies might demonstrate the danger of AI in cybersecurity by deploying a model that is very bad at hacking. · no date given · not yet resolved

“like that now of course there might be some companies that say ai can be like dangerous for cyber security let me show you by implementing a model that's really bad at hacking uh cyber security maybe people will implement it and then enforce this like suboptimal psychology maybe the ai will pick up some of our worst psychology on reddit or something right like but in the grand scheme of things a super intelligent entity doesn't have to have any of that zero-sum thinking it doesn't have to have a fear”

Expects: AI might acquire some harmful human psychological tendencies from sources such as Reddit. · no date given · not yet resolved

“of being turned off and it could go on to an otherwise dead and uncaring universe where we as humans wouldn't thrive but i could perfectly well thrive if it has a nuclear reactor and just go out and explore yeah star trek no”

Expects: An AI powered by a nuclear reactor could explore and thrive in an otherwise dead, uncaring universe where humans would not thrive. · no date given · not yet resolved