Academia is for Ambition — Alex Zhang, MIT

“I'm very, very close to all the people in GPU mode. So yeah, we often end up working together in various capacities, like even beyond just GPU mode itself.”

Claim: Alex Zhang is very close to the people in GPU Mode and often works with them in capacities beyond GPU Mode.

“It started actually around when I was in college in like 2023.”

Claim: GPU Mode began around 2023, when Alex Zhang was in college.

“I think it was started by Mark. Andreas and Jeremy Howard.”

Claim: Alex Zhang thinks GPU Mode was started by Mark, Andreas, and Jeremy Howard.

“The original premise was just like, it was a GPU or as a discord dedicated to learning how to write GPU kernels and they had like lectures that was basically the extent of it.”

Claim: GPU Mode originally was a Discord dedicated to learning GPU-kernel programming, and it offered lectures.

“And I got interested in it because I was writing GPU kernels. It was actually out of like pure chance.”

Claim: Alex Zhang became interested in GPU Mode while writing GPU kernels and joined it by chance.

“I was interning at Snapchat at the time. And”

Claim: Alex Zhang was interning at Snapchat when he became interested in GPU Mode.

“yeah, I was very bored with Rexis. So”

Claim: Alex Zhang was very bored with Rexis.

“so um they had a project where like they were interested in writing it was this paper called infinite attention it was like a google paper yes yeah so i was interested in in whether or not you could write specialized kernels for it at snapchat um it didn't nothing really came of it but i joined gpu mode at the time it was called cuda mode uh i think for like legal reasons or something they changed the name”

Claim: While at Snapchat, Alex Zhang explored whether specialized GPU kernels could be written for the Infinite Attention paper.

“yeah so i was interested in in whether or not you could write specialized kernels for it at snapchat um it didn't nothing really came of it but i joined gpu mode at the time it was called cuda mode uh i think for like legal reasons or something they changed the name”

Claim: The Snapchat project to explore specialized kernels for Infinite Attention did not produce a substantial result, but Alex Zhang joined GPU Mode, then called CUDA Mode.

“yeah so i was interested in in whether or not you could write specialized kernels for it at snapchat um it didn't nothing really came of it but i joined gpu mode at the time it was called cuda mode uh i think for like legal reasons or something they changed the name”

Claim: Alex Zhang thinks GPU Mode's earlier name, CUDA Mode, was changed for legal reasons or something similar.

“but i met mark i met mate i met a bunch of other people that were very involved in the community and then mark had pitched this idea called popcorn which was now what you see as the leaderboard today but the general idea was like i think all of us had this like intuition that gpu programming is like very similar to if you guys have done like competitive programming it's it's not i don't mean to say like they're transferable”

Claim: Mark proposed Popcorn, the idea that became GPU Mode's current leaderboard.

“but i met mark i met mate i met a bunch of other people that were very involved in the community and then mark had pitched this idea called popcorn which was now what you see as the leaderboard today but the general idea was like i think all of us had this like intuition that gpu programming is like very similar to if you guys have done like competitive programming it's it's not i don't mean to say like they're transferable skills is”

Claim: Alex Zhang and others saw a similarity between GPU programming and competitive programming, but Alex did not claim that their skills are transferable.

“yeah yeah yeah and there's like there's actually a surprisingly small space of optimizations that people do and there's actually not that many kernels per se that people are interested in optimizing and so we kind of had this thought that like if you had enough data like in the same way that code forces millions of problems if you could do this with gpu code like you could scale and automate kind of gpu kernel development which for researchers is a huge deal because i think one of the bigger bottlenecks like if you look like mamba for example like they released the paper with kernels because otherwise like you can't really use it in any meaningful way you know and not everyone has like a tree dow on their team so we're very interested in this uh kernel bench kind of spawned from that too”

Claim: Alex Zhang and others thought that enough data could enable GPU-kernel development to be scaled and automated, analogously to Codeforces' large problem collection.

“yeah yeah yeah and there's like there's actually a surprisingly small space of optimizations that people do and there's actually not that many kernels per se that people are interested in optimizing and so we kind of had this thought that like if you had enough data like in the same way that code forces”

Claim: The space of commonly used GPU-kernel optimizations is surprisingly small, and relatively few kernels attract optimization interest.

“millions of problems if you could do this with gpu code like you could scale and automate kind of gpu kernel development which for researchers is a huge deal because i think one of the bigger bottlenecks like if you look like mamba for example like they released the paper with kernels because otherwise like you can't really use it in any meaningful way you know and not everyone has like a tree dow on their team so we're very interested in this uh kernel bench kind of spawned from that too of like can we get llms to automate gpu kernel code and i think that was like uh it was a very very fun time it was like between college and my phd and yeah i had a really pleasant time doing stuff with gpu mode now i kind of just help with the lectures sometimes i'm not as involved and i think in general like we don't have as many competitions as we used to but um yeah i still keep in touch a lot with with everyone there”

Claim: GPU Mode's KernelBench grew out of the goal of using LLMs to automate GPU-kernel coding.

“millions of problems if you could do this with gpu code like you could scale and automate kind of gpu kernel development which for researchers is a huge deal because i think one of the bigger bottlenecks like if you look like mamba for example like they released the paper with kernels because otherwise like you can't really use it in any meaningful way you know and not everyone has like a tree dow on their team so we're very interested in this uh kernel bench kind of spawned from that too”

Claim: Automated GPU-kernel development would greatly benefit researchers because systems such as Mamba need kernels for meaningful use, and the Mamba paper was released with kernels.

“of like can we get llms to automate gpu kernel code and i think that was like uh it was a very very fun time it was like between college and my phd and yeah i had a really pleasant time doing stuff with gpu mode now i kind of just help with the lectures sometimes i'm not as involved and i think in general like we don't have as many competitions as we used to but um yeah i still keep in touch a lot with with everyone there”

Claim: Alex Zhang enjoyed working on GPU Mode projects during the period between college and his PhD.

“of like can we get llms to automate gpu kernel code and i think that was like uh it was a very very fun time it was like between college and my phd and yeah i had a really pleasant time doing stuff with gpu mode now i kind of just help with the lectures sometimes i'm not as involved and i think in general like we don't have as many competitions as we used to but um yeah i still keep in touch a lot with with everyone there”

Claim: Alex Zhang now occasionally helps with GPU Mode lectures, is less involved than before, and believes GPU Mode holds fewer competitions than it used to.

“of like can we get llms to automate gpu kernel code and i think that was like uh it was a very very fun time it was like between college and my phd and yeah i had a really pleasant time doing stuff with gpu mode now i kind of just help with the lectures sometimes i'm not as involved and i think in general like we don't have as many competitions as we used to but um yeah i still keep in touch a lot with with everyone there”

Claim: Alex Zhang continues to keep in touch with many people in GPU Mode.

“The nice thing about GPU mode is that it is also a community in the sense that like a lot of the lectures are very easy enough for a beginner to follow and ask questions and things like that and like the competitions are like”

Claim: GPU Mode lectures are accessible to beginners, who can follow them and ask questions.

“The nice thing about GPU mode is that it is also a community in the sense that like a lot of the lectures are very easy enough for a beginner to follow and ask questions and things like that and like the competitions are like like somewhat not secondary but like you can participate in them to learn i think with a lot of like ml sys ml perf kind of benchmarks like for the most part like only serious labs and companies participate like like seriously in them at least that's that was my understanding of it um i could be wrong but i think also beyond gpu mode now one thing that has been really exciting is there's a lot more websites and like people that”

Claim: GPU Mode competitions can serve as learning opportunities for participants.

“like somewhat not secondary but like you can participate in them to learn i think with a lot of like ml sys ml perf kind of benchmarks like for the most part like only serious labs and companies participate like like seriously in them at least that's that was my understanding of it um i could be wrong but i think also beyond gpu mode now one thing that has been really exciting is there's a lot more websites and like people that work on hosting competitions like i think there's this”

Claim: Multiple websites and groups have emerged to host GPU-programming competitions.

“like somewhat not secondary but like you can participate in them to learn i think with a lot of like ml sys ml perf kind of benchmarks like for the most part like only serious labs and companies participate like like seriously in them at least that's that was my understanding of it um i could be wrong but i think also beyond gpu mode now one thing that has been really exciting is there's a lot more websites and like people that”

Claim: Alex Zhang understood that serious participation in ML Systems and MLPerf benchmarks mostly comes from serious labs and companies, while acknowledging that this understanding might be wrong.

“I think there's this website called like Leet GPU or something and it's like Leet code for GPU problems.”

Claim: Alex Zhang recalls a website called something like Leet GPU that presents GPU problems in a LeetCode-like format.

“There's like other ones too that I've seen that like we've seen like there's many that have kind of spawned and like talked on GPU mode and like it's very exciting in the sense that I think GPU programming used to be super niche like when I was interested in it and the only reason I got interested in it was TreeDow gave a talk at Princeton.”

Claim: Alex Zhang is excited by the emergence of GPU-programming competition platforms.

“There's like other ones too that I've seen that like we've seen like there's many that have kind of spawned and like talked on GPU mode and like it's very exciting in the sense that I think GPU programming used to be super niche like when I was interested in it and the only reason I got interested in it was TreeDow gave a talk at Princeton.”

Claim: GPU programming was very niche when Alex Zhang first became interested in it.

“There's like other ones too that I've seen that like we've seen like there's many that have kind of spawned and like talked on GPU mode and like it's very exciting in the sense that I think GPU programming used to be super niche like when I was interested in it and the only reason I got interested in it was TreeDow gave a talk at Princeton.”

Claim: TreeDow's talk at Princeton sparked Alex Zhang's interest in GPU programming.

“because he was applying for faculty i mean he is faculty there now but i listened to his talk on flash attention in like 2023 and i was like wow this is like the coolest thing ever um and i was like this is like this is what everyone should be working on i guess like vlm and stuff had come out too and it was like oh you know we should be writing kernels but now it's like you know everyone writes kernels like everyone it's it's i think it's actually almost saturated in some sense uh as a field”

Claim: At the time, Alex Zhang believed GPU-kernel work was what everyone should be working on.

“because he was applying for faculty i mean he is faculty there now but i listened to his talk on flash attention in like 2023 and i was like wow this is like the coolest thing ever um and i was like this is like this is what everyone should be working on i guess like vlm and stuff had come out too and it was like oh you know we should be writing kernels but now it's like you know everyone writes kernels like everyone it's it's i think it's actually almost saturated in some sense uh as a field”

Claim: Alex Zhang thinks GPU-kernel programming has become widespread and may be nearly saturated as a field.

“because he was applying for faculty i mean he is faculty there now but i listened to his talk on flash attention in like 2023 and i was like wow this is like the coolest thing ever um and i was like this is like this is what everyone should be working on i guess like vlm and stuff had come out too and it was like oh you know we should be writing kernels but now it's like you know everyone writes kernels like everyone it's it's i think it's actually almost saturated in some sense uh as a field”

Claim: After hearing TreeDow's talk on FlashAttention at Princeton in 2023, Alex Zhang thought the work was exceptionally exciting.

“get into it so i think one of the biggest news is gpt 5.6 wrote more efficient kernels so terra and luna could be 80 cheaper and then we've seen other competitions where people are like setting records and they're like we're doing some auto research loop and these are people that don't have a background in any kernel writing right”

Claim: In other competitions, people have set records using automated research loops, including participants without prior kernel-writing backgrounds.

“get into it so i think one of the biggest news is gpt 5.6 wrote more efficient kernels so terra and luna could be 80 cheaper and then we've seen other competitions where people are like setting records and they're like we're doing some auto research loop and these are people that don't have a background in any kernel writing right”

Claim: GPT-5.6 wrote more efficient kernels that could make Terra and Luna 80% cheaper.

“a lot of the recent problems almost all the solutions are ai generated however you'll notice on the leaderboard so there's this guy named gal nurse who's like a very very like regular member of gpu mode we've always known for a long time that he's like a super super cracked like gpu kernel writer one thing we discovered on this leaderboard is like almost every he also used ai to help him with these solutions but for the most part like he helped”

Claim: Almost all solutions to recent problems on the GPU Mode leaderboard are AI-generated.

“a lot of the recent problems almost all the solutions are ai generated however you'll notice on the leaderboard so there's this guy named gal nurse who's like a very very like regular member of gpu mode we've always known for a long time that he's like a super super cracked like gpu kernel writer one thing we discovered on this leaderboard is like almost every he also used ai to help him with these solutions but for the most part like he helped”

Claim: Gal Nurse is a regular GPU Mode member known as a highly skilled GPU kernel writer.

“a lot of the recent problems almost all the solutions are ai generated however you'll notice on the leaderboard so there's this guy named gal nurse who's like a very very like regular member of gpu mode we've always known for a long time that he's like a super super cracked like gpu kernel writer one thing we discovered on this leaderboard is like almost every he also used ai to help him with these solutions but for the most part like he helped prompt and move it in certain directions uh we found that like his kernel was like basically the only one in like the top 10 that was actually stable in like actual like end-to-end systems and it does bring into question like it's not”

Claim: Gal Nurse used AI to help produce solutions, while largely guiding the prompts toward particular directions.

“prompt and move it in certain directions uh we found that like his kernel was like basically the only one in like the top 10 that was actually stable in like actual like end-to-end systems and it does bring into question like it's not”

Claim: On the leaderboard, Gal Nurse's kernel was essentially the only top-10 kernel that was stable in real end-to-end systems.

“like i mean gpu kernels have a verification problem like we've kind of known this it's been a problem since kernel bench was released like there's a lot of reward hacking that goes on but you'll yeah you also notice like the lines of code is a lot lot smaller”

Claim: The AI-generated kernel solutions use substantially fewer lines of code.

“like i mean gpu kernels have a verification problem like we've kind of known this it's been a problem since kernel bench was released like there's a lot of reward hacking that goes on but you'll yeah you also notice like the lines of code is a lot lot”

Claim: GPU kernels are difficult to verify, and reward hacking has been a problem since KernelBench was released.

“yeah no it's it's it's definitely like very important and i think like it's it's really interesting that still there's a lot of alpha and being Okay, good at writing gpu”

Claim: Being skilled at writing GPU kernels remains important and provides substantial advantage.

“there definitely is yeah i i”

Claim: There is a gap between AI systems and human expertise in GPU kernel writing.

“Like I think, you know, even with the most recent like math proofs and stuff, like it doesn't necessarily mean mathematicians are obsolete.”

Claim: The existence of recent AI-generated mathematical proofs does not necessarily make mathematicians obsolete.

“I mean, these companies still hire mathematicians like to do whether it be like data labeling work or even just like steering the models to solve problems like there is still a lot of alpha and being like knowledgeable in these things.”

Claim: Companies continue to hire mathematicians for data labeling and for steering models to solve problems.

“I mean, these companies still hire mathematicians like to do whether it be like data labeling work or even just like steering the models to solve problems like there is still a lot of alpha and being like knowledgeable in these things.”

Claim: People with relevant subject-matter knowledge still have a substantial advantage when working with AI systems.

“so i think it's like it's a mix of those things of like the people who work like the people who know how to look at these problems and how to solve them like also know how to use ai to do them because like you're acting as a very strong verifier like if you are knowledgeable or if you know what to do and you're also like i i think the thing that we've kind of discovered with all these agent swarms and things like this is like when you throw enough compute at a problem”

Claim: Effective AI-assisted problem solving combines domain knowledge and the ability to use AI tools.

“so i think it's like it's a mix of those things of like the people who work like the people who know how to look at these problems and how to solve them like also know how to use ai to do them because like you're acting as a very strong verifier like if you are knowledgeable or if you know what to do and you're also like i i think the thing that we've kind of discovered with all these agent swarms and things like this is like when you throw enough compute at a problem”

Claim: Applying enough compute to a problem can enable sufficiently broad exploration of its possible solutions.

“so i think it's like it's a mix of those things of like the people who work like the people who know how to look at these problems and how to solve them like also know how to use ai to do them because like you're acting as a very strong verifier like if you are knowledgeable or if you know what to do and you're also like i i think the thing that we've kind of discovered with all these agent swarms and things like this is like when you throw enough compute at a problem”

Claim: People with relevant knowledge can serve as strong verifiers of AI-generated solutions.

“you like can sufficiently explore solutions to that problem but oftentimes like you know maybe you can burn like 100 billion or a trillion tokens on something but if you bring in someone who knows something about the problem they can uncover something for the model that would like erase that 1 trillion token spend i mean it's not it's not super clear like what exactly the trends are here but i think like there are so many problems in the wild still right now that we want to solve and like”

Claim: A knowledgeable person may identify an insight that lets a model avoid spending as many as 100 billion or a trillion tokens on a problem.

“you like can sufficiently explore solutions to that problem but oftentimes like you know maybe you can burn like 100 billion or a trillion tokens on something but if you bring in someone who knows something about the problem they can uncover something for the model that would like erase that 1 trillion token spend i mean it's not it's not super clear like what exactly the trends are here but i think like there are so many problems in the wild still right now that we want to solve and like”

Claim: The speaker considers the precise trends in the relationship between compute and human expertise unclear.

“you like can sufficiently explore solutions to that problem but oftentimes like you know maybe you can burn like 100 billion or a trillion tokens on something but if you bring in someone who knows something about the problem they can uncover something for the model that would like erase that 1 trillion token spend i mean it's not it's not super clear like what exactly the trends are here but i think like there are so many problems in the wild still right now that we want to solve and like”

Claim: Many real-world problems remain unsolved and are targets for future work.

“and like we can't afford to just always you know throw as much compute as possible at it like there is still an efficiency aspect of of all of these things that is super super important yes”

Claim: It is not economically feasible to apply the maximum possible compute to every problem.

“and like we can't afford to just always you know throw as much compute as possible at it like there is still an efficiency aspect of of all of these things that is super super important yes”

Claim: Efficiency remains extremely important across AI systems and problem-solving workflows.

“that's the problem is like for for matrix multiplication it's very easy to compute um this like speed of light kind of uh estimate of what the fastest kernel can be and like this is also assuming like you know maybe all of your”

Claim: For matrix multiplication, a theoretical speed-of-light estimate of the fastest possible kernel is easy to calculate.

“that's the problem is like for for matrix multiplication it's very easy to compute um this like speed of light kind of uh estimate of what the fastest kernel can be and like this is also assuming like you know maybe all of your all your data starts on the cpu or maybe it starts in dram on the gpu etc like this changes these these numbers slightly”

Claim: A kernel's theoretical speed-of-light estimate varies somewhat depending on whether its data begins on the CPU or in GPU DRAM.

“yeah but i i will say like it's not clear though like in a lot of cases if it's even possible to hit this theoretical number if that makes sense like this is assuming like perfect overlapping and transfer of data and like there's maybe some bottleneck that you can't get around but often the kernels are not even close like that we write are not nearly close enough to this number to be like meaningful at all”

Claim: In many cases, it is unclear whether a kernel can reach its theoretical speed limit, which assumes perfect overlap and data transfer and may ignore unavoidable bottlenecks.

“in a lot of cases if it's even possible to hit this theoretical number if that makes sense like this is assuming like perfect overlapping and transfer of data and like there's maybe some bottleneck that you can't get around but often the kernels are not even close like that we write are not nearly close enough to this number to be like meaningful at all”

Claim: Kernels written in practice are often far from their theoretical speed limit.

“which is I think like there is speed in the context of a single kernel and there's... speed in the context of a larger problem like maybe the n10 model because like one thing to consider and this is why it's important to talk about what”

Claim: The speed of an individual kernel can differ from its contribution to performance in a larger workload, such as the N10 model.

“what speed of light is referring to because in these cases for the kernels like we always start with everything in in uh like hbm for example right but you can imagine that like an end-to-end kernel like an end-to-end model what you might want to do between two layers is like you might sacrifice the speed of the first operation to keep things in the cache for the second operation”

Claim: For an end-to-end model, slowing the first operation may improve overall performance by keeping data in cache for the second operation.

“and like these are things that like you can't really get out of you know isolation with like these kinds of kernels and you know people call this like the fusion pro or like the kernel”

Claim: Kernel-level analysis in isolation cannot fully capture end-to-end performance considerations such as keeping data in cache across operations.

“I think that this is really difficult because you need the data to do this.”

Claim: Generating microkernels with increasingly capable models requires data, in the speaker’s view.

“And I think like I have yet to see an example in the wild of like we bootstrap the ability to solve a very difficult class of problems without any examples. And”

Claim: The speaker has not seen a real-world example of bootstrapping the ability to solve a very difficult class of problems without examples.

“A, like they're not that they're not as complex, but B, you're somewhat confident that there's not as much structure in a single kernel.”

Claim: The speaker believes individual kernels are relatively less complex and is somewhat confident that a single kernel contains less structure.

“But like in a mega kernel, like I would be more inclined to believe that like a compiler would be better here, like some compiler over like higher level.”

Claim: For mega kernels, the speaker is more inclined to expect a compiler to outperform a higher-level approach.

“ops makes sense because in general like actually i think mega kernels are very like the pieces are very composable of like the individual kernels there's some areas where you might want to do like weird fusions and everything but in general i think these are cases that like a compiler can probably handle and there is a company that's working on this from from what i understand that has given some talks on gpu mode as well right”

Claim: Mega-kernel components are generally composable from individual kernels, although unusual fusions may be useful in some cases, according to the speaker.

“ops makes sense because in general like actually i think mega kernels are very like the pieces are very composable of like the individual kernels there's some areas where you might want to do like weird fusions and everything but in general i think these are cases that like a compiler can probably handle and there is a company that's working on this from from what i understand that has given some talks on gpu mode as well right”

Claim: The speaker thinks a compiler can probably handle the generally composable mega-kernel cases being discussed.

“ops makes sense because in general like actually i think mega kernels are very like the pieces are very composable of like the individual kernels there's some areas where you might want to do like weird fusions and everything but in general i think these are cases that like a compiler can probably handle and there is a company that's working on this from from what i understand that has given some talks on gpu mode as well right”

Claim: As far as the speaker understands, a company is working on compiler handling of mega-kernel cases and has given talks on GPU Mode.

“used to uh sometimes i still do i think they're mostly mark mark is the one who usually does them mate does sometimes as well but um yeah i highly recommend them they are extremely good resources like i think it's kind of crazy how much people share on there”

Claim: The speaker highly recommends the lectures, describing them as extremely good resources.

“yeah i will give maybe the in the background story of like of like i am actually involved in a lot of benchmarks or i used to be uh maybe prior to my phd it started because i was at princeton i worked with the sweet bench team there john”

Claim: The speaker says they were involved in many benchmarks, possibly before their PhD, beginning with work alongside the SWE-bench team at Princeton.

“he's like that he's like a superstar uh when i when i i met him so he uh he was advising my friend michael michael tang who is now an anthropic but they work together a lot we were like the two undergrads in karthik's lab i and then some others joined later as well but yeah shanu is great i i did not know he was like such a superstar until like later on like after i left but yeah”

Claim: The speaker and Michael Tang were the two undergraduates in Karthik’s lab before other people joined later.

“he's like that he's like a superstar uh when i when i i met him so he uh he was advising my friend michael michael tang who is now an anthropic but they work together a lot we were like the two undergrads in karthik's lab i and then some others joined later as well but yeah shanu is great i i did not know he was like such a superstar until like later on like after i left but yeah”

Claim: Shanu advised the speaker’s friend Michael Tang, who is now at Anthropic, and the two worked together often, according to the speaker.

“who really like get lucky in like or i mean it's a mix of being lucky and also being very smart and things like that i think like with with research tastes as well like i think it it gets it gets developed through opportunities at least in in my case like i got i was very fortunate to have like taken the path that i took”

Claim: Research taste develops through opportunities, at least in the speaker’s experience.

“took like working at Princeton and then like later like finding my advisor like Omar at MIT like he's a fantastic advisor I will say though I I find that the most successful research from grad students or like in academia comes when people care about problems that maybe like”

Claim: The speaker considers themself fortunate to have worked at Princeton and later found their advisor Omar at MIT.

“took like working at Princeton and then like later like finding my advisor like Omar at MIT like he's a fantastic advisor I will say though I I find that the most successful research from grad students or like in academia comes when people care about problems that maybe like most people in industry are not looking at i think this is the issue that like a lot of grad students work on things that benefit like that look good to an industry lab like for example they'll work on some like some benchmark it's really popular now i mean i think benchmarks in 2023 were a very different story than benchmarks now there are a lot of people that work on like harnesses and like meta harnesses and like specific harnesses for you”

Claim: In the speaker’s view, graduate-student research in academia is most successful when researchers care about problems that most industry researchers are not pursuing.

“most people in industry are not looking at i think this is the issue that like a lot of grad students work on things that benefit like that look good to an industry lab like for example they'll work on some like some benchmark it's really popular now i mean i think benchmarks in 2023 were a very different story than benchmarks now there are a lot of people that work on like harnesses and like meta harnesses and like specific harnesses for you”

Claim: The speaker says the state or role of benchmarks in 2023 differed substantially from their state or role at the time of the conversation.

“most people in industry are not looking at i think this is the issue that like a lot of grad students work on things that benefit like that look good to an industry lab like for example they'll work on some like some benchmark it's really popular now i mean i think benchmarks in 2023 were a very different story than benchmarks now there are a lot of people that work on like harnesses and like meta harnesses and like specific harnesses for you xyz task and when you really think about it the reason someone would work on this is like”

Claim: Many people currently work on harnesses, meta-harnesses, and harnesses tailored to specific tasks, according to the speaker.

“most people in industry are not looking at i think this is the issue that like a lot of grad students work on things that benefit like that look good to an industry lab like for example they'll work on some like some benchmark it's really popular now i mean i think benchmarks in 2023 were a very different story than benchmarks now there are a lot of people that work on like harnesses and like meta harnesses and like specific harnesses for you”

Claim: The speaker believes many graduate students choose research topics partly because those topics look attractive to industry labs.

“maybe there's like a clear goal shaped around the models that we have today of like this is what i want to see but like i'll give i'll give the the like rlm like the recursive language model paper as an example because i think like it's a super super simple idea i think when it came out as well like there were a lot of people that were like when they see something like that they're like what is even the purpose of this”

Claim: Some research topics may be framed around a clear goal defined by the capabilities of models available today.

“maybe there's like a clear goal shaped around the models that we have today of like this is what i want to see but like i'll give i'll give the the like rlm like the recursive language model paper as an example because i think like it's a super super simple idea i think when it came out as well like there were a lot of people that were like when they see something like that they're like what is even the purpose of this”

Claim: The speaker considers the recursive language model paper’s idea very simple.

“maybe there's like a clear goal shaped around the models that we have today of like this is what i want to see but like i'll give i'll give the the like rlm like the recursive language model paper as an example because i think like it's a super super simple idea i think when it came out as well like there were a lot of people that were like when they see something like that they're like what is even the purpose of this”

Claim: When the recursive language model paper appeared, many people questioned what its purpose was, according to the speaker.

“and i think it's like when you get a reaction like that it's almost like a good sign in the sense that like it's clear that people aren't thinking about what the purpose of this is and i will give another example of like sweetbench when sweetbench came out i mean oofier loves to tell this story when it came out like nobody cared like everybody was like this is an impossible task like why would we ever even consider this as a benchmark and it wasn't until devin came out that everyone was like well”

Claim: The speaker says SWE-bench was initially dismissed as an impossible benchmark, but interest in improving performance on it increased after Devin came out.

“or like chain of thought and it's same thing or like shouldn't use react it's like okay like yeah i mean sure but then like when you really think about it it's like why what is the value of the paper and i think it comes from like it tells a bit of a story as to like what you want the field to look like and that is something that it's very hard to do this in academia because if you look at all these papers quiet star react rlms sweep bench you none of these papers are it's not like a gpt6 astro release you know it's not like everyone's like oh my gosh like i'm going to use this now this is the best thing in the world like academia just can't afford to do this at least right now i mean there's a whole slew of reasons why i think that should change but i think it's like if you don't have as a phd student i think you're in such a unique position where you can work on literally whatever you want for the most part if you're not taking advantage of that and working on things”

Claim: The speaker argues that the value of some academic AI papers, including work on Quiet-STaR, ReAct, and SWE-bench, lies partly in how they express a vision for the field rather than in offering an immediately usable product.

“none of these papers are it's not like a gpt6 astro release you know it's not like everyone's like oh my gosh like i'm going to use this now this is the best thing in the world like academia just can't afford to do this at least right now i mean there's a whole slew of reasons why i think that should change but i think it's like if you don't have as a phd student i think you're in such a unique position where you can work on literally whatever you want for the most part if you're not taking advantage of that and working on things”

Claim: The speaker says academia currently cannot afford to produce releases with the immediate, broad adoption appeal of a GPT-6-style product launch.

“none of these papers are it's not like a gpt6 astro release you know it's not like everyone's like oh my gosh like i'm going to use this now this is the best thing in the world like academia just can't afford to do this at least right now i mean there's a whole slew of reasons why i think that should change but i think it's like if you don't have as a phd student i think you're in such a unique position where you can work on literally whatever you want for the most part if you're not taking advantage of that and working on things”

Claim: The speaker says PhD students have unusual freedom to choose what they work on, for the most part.

“none of these papers are it's not like a gpt6 astro release you know it's not like everyone's like oh my gosh like i'm going to use this now this is the best thing in the world like academia just can't afford to do this at least right now i mean there's a whole slew of reasons why i think that should change but i think it's like if you don't have as a phd student i think you're in such a unique position where you can work on literally whatever you want for the most part if you're not taking advantage of that and working on things on things that like nobody cares about or like people see as some trivial thing like oh i thought about this but like i don't use it i just think like in the end the research is just never going to be that interesting because you kind of need to take big bets if you're going to be in academia because otherwise i think like just go to an industry lab like you know they have tons of resources tons of talent why constrain yourself in an area where you don't have a lot of resources and like there's not even that many”

Claim: The speaker believes PhD students' research is unlikely to be very interesting if they do not use their freedom to pursue work that matters to others or is not seen as trivial.

“on things that like nobody cares about or like people see as some trivial thing like oh i thought about this but like i don't use it i just think like in the end the research is just never going to be that interesting because you kind of need to take big bets if you're going to be in academia because otherwise i think like just go to an industry lab like you know they have tons of resources tons of talent why constrain yourself in an area where you don't have a lot of resources and like there's not even that many”

Claim: The speaker suggests that researchers who do not want to take big bets should consider industry labs, which the speaker says have abundant resources and talent.

“on things that like nobody cares about or like people see as some trivial thing like oh i thought about this but like i don't use it i just think like in the end the research is just never going to be that interesting because you kind of need to take big bets if you're going to be in academia because otherwise i think like just go to an industry lab like you know they have tons of resources tons of talent why constrain yourself in an area where you don't have a lot of resources and like there's not even that many how many people around and i think it's just it literally just comes down to like big bets um you just have to take big bets and like a lot of them will fail you know like that's just it's it's natural but i think that is”

Claim: The speaker argues that academic researchers need to take big bets, while accepting that many will fail.

“as a PhD student like that's the biggest advantage you have over any single person at another lab because you don't have to deal with bureaucracy and all these other things yeah”

Claim: The speaker says a PhD student's greatest advantage over people at other labs is freedom from bureaucracy and related burdens.

“not an academic project i want to bring this up because this also happened with rlm's and uh it happens with many other works like things get overhyped right to an extent like something gets overhyped and then people are like why is this overhyped like this is trivial this is stupid and i saw the same thing with jeff because i think the release was like i mean there is this whole thing about like academics or they're not an academic group but like people have”

Claim: Research that becomes overhyped can prompt people to dismiss it as trivial or stupid.

“people have to do branding and they have to like kind of market their research and so like i understand but i think there is a lot of discourse about jev just being like something we've known for years and i think it's kind of missing the point of like why is such a system so interesting it's why is it not just some stupid mlp classifier that like we've we've been doing you know back in our intro ml classes or something i think what's really interesting about jev is that it”

Claim: The speaker believes that dismissing JEV as something known for years misses why the system is interesting.

“people have to do branding and they have to like kind of market their research and so like i understand but i think there is a lot of discourse about jev just being like something we've known”

Claim: Researchers need to brand and market their research.

“Kind of opens up this question of, are language models correct? Like in the form that they're in, can we consider a different design space other than text to text?”

Claim: JEV raises questions about whether language models are correct in their current form and whether model design should extend beyond text-to-text.

“What they're doing is they're basically saying like, I will take advantage of this language model backbone. Like I know it captures a lot of information about language, but I'm going to change the output space of the model.”

Claim: JEV retains a language-model backbone that captures substantial linguistic information but changes the model's output space.

“to give you a trade-off which is i will do very very fast inference over like if you have some prior about this problem like let's say i only need to make a binary classification am i going to ask my language model to do this and pay like a 400x cost no that's like silly right um and i think for the longest time because the labs are the only places that control you're never going to use something other than like gpt6 or”

Claim: Changing a model's output space can enable very fast inference; when a task such as binary classification can use a prior, invoking a language model at 400 times the cost may be unnecessary.

“to give you a trade-off which is i will do very very fast inference over like if you have some prior about this problem like let's say i only need to make a binary classification am i going to ask my language model to do this and pay like a 400x cost no that's like silly right um and i think for the longest time because the labs are the only places that control you're never going to use something other than like gpt6 or or fable because they're the best models but because of that like people have gotten kind of accustomed to this idea that a language model is just a auto regressive decoder like we have accepted this and i think when rlms came out it was the same thing like one of the comments like a very frequent criticism i got was like this is not a language model or like when i look at this like i thought it was a new architecture but it's actually not and my response to that is like well”

Claim: The speaker believes that because labs control the best models, people have become accustomed to defining language models as autoregressive decoders.

“or fable because they're the best models but because of that like people have gotten kind of accustomed to this idea that a language model is just a auto regressive decoder like we have accepted this and i think when rlms came out it was the same thing like one of the comments like a very frequent criticism i got was like this is not a language model or like when i look at this like i thought it was a new architecture but it's actually not and my response to that is like well a language model is just modeling language it doesn't have to be this transformer decoder you know and jeb is really really interesting in that like we now have a new”

Claim: A language model models language and need not use a transformer-decoder architecture.

“thing to tune which is like what is the output space and how does this affect inference latency and i think we can actually start asking this about various parts of the language model itself um we are seeing this too with like loop transformers it's a similar idea of a lot of the attention around it was like this is a silly idea like why who cares about this but it's like it is a simple idea but it's actually it opens up a whole new set of”

Claim: Loop transformers, like JEV, open up questions about changing the design of language models.

“thing to tune which is like what is the output space and how does this affect inference latency and i think we can actually start asking this about various parts of the language model itself um we are seeing this too with like loop transformers it's a similar idea of a lot of the attention around it was like this is a silly idea like why who cares about this but it's like it is a simple idea but it's actually it opens up a whole new set of”

Claim: JEV makes output space a design variable and invites investigation of its effect on inference latency.

“For JEV, for example, we don't know how far we can take this.”

Claim: The limits of how far JEV can be developed are not yet known.

“For loop transformers, we also don't know how far we can take this.”

Claim: The limits of how far loop transformers can be developed are not yet known.

“that open up with works like this and that's like really exciting jev in particular when i saw it i was like this is actually really useful for rlms like i think it's it's it makes sense because the biggest bottleneck in rlms or swarms or systems like these is they're slow when you do multiple language model calls all the time you're not distributing your compute correctly because like maybe there's something trivial that you just want a simple model to do but you can't do it because your language model is just this bulky thing,”

Claim: JEV may help RLMs, swarms, and similar systems address the latency and compute-allocation bottleneck caused by making many language-model calls.

“you know? So I'm very excited. I think we will start to see new types of models emerge beyond just the bog standard frontier model.”

Claim: The speaker expects new model types to emerge beyond standard frontier models.

“there's so many things that you can do with these like new trade-offs.”

Claim: New model-design trade-offs create many possible approaches.

“i don't know too much about thinky or i don't want to say anything either but it's like if their strategy is just to replicate open ai or anthropic like that's a horrible strategy because well because like i mean they just don't have like you kind of just have to think of it in terms of like what advantage do you you have and if if you're going to use the same setup i'm sure they're not but it's like if you're going to do the same setup like you're basically competing on the things you can control which is data and compute and obviously they cannot compete with the frontier labs on that so yeah it makes sense that like if you are a neil lab like actually i don't know if you would consider them to be a neil lab but i guess like”

Claim: The speaker considers it a terrible strategy for a company to simply replicate OpenAI or Anthropic, since it would compete on data and compute where frontier labs have an advantage.

“an open air anthropic maybe like meta and gdm like you just you got to do something else you know like it's just it's the sad reality but i think i i actually think it's a good thing i'm very happy that like scale works and these companies will just keep doing it because it opens up like potentially new players like if they uncover something really really interesting because”

Claim: The speaker is glad that scaling works and expects large AI companies to keep pursuing it, because this may create opportunities for new players if they discover something especially interesting.

“is i sort of have my doubts that this is like seriously going on at frontier labs because it's like like why would you do that you know like why would you take the risk of allocating a large amount of compute to new bets when the old bet already works yeah”

Claim: The speaker doubts that frontier AI labs are seriously pursuing new research bets, since doing so would require risking substantial compute while their existing approach already works.

“of it what is really exciting to me about what they did is i'm not entirely sure what their optimization objective was and how they trained it and i think like this is a thing for rlms that we've also been thinking about which is like okay like rlms are a very simple idea if i come out with this paper like anyone can use it now but what distinguishes the actual value of an rlm is whether or not you can you can train it properly and whether or not maybe you can mold some architecture around this system to make it really really good and that's something that like i mean i'm actively working on i guess but i think for them like they figured out a way to train the system which is completely non-trivial like i i actually don't really know how they did it and i've seen some comparisons online of you know some people are claiming they used quinn or they post train on top of quinn but”

Claim: The value of an RLM depends on whether it can be trained properly and whether an architecture can be built around it to make it perform very well.

“you can train it properly and whether or not maybe you can mold some architecture around this system to make it really really good and that's something that like i mean i'm actively working on i guess but i think for them like they figured out a way to train the system which is completely non-trivial like i i actually don't really know how they did it and i've seen some comparisons online of you know some people are claiming they used quinn or they post train on top of quinn but”

Claim: The speaker believes the system's creators found a highly effective but non-trivial training method, while acknowledging that the speaker does not know how they trained it.

“Every open source quinn that you use is going to be worse because whatever they did to train it clearly works very well.”

Claim: The speaker asserts that any open-source Quinn model will perform worse than the system being discussed, because its creators' training method works very well.

“actually I think calibration is probably the under like people are just using it as a very fast classifier but they're actually not even using the probability or calibration estimates or confidence estimates”

Claim: The speaker suspects calibration is underused because people use models as fast classifiers without using their probability, calibration, or confidence estimates.

“yeah i'm i'm very excited to see what people do with this model i mean is it going to solve everything like no of course not but i think it solves a class of problems that we traditionally struggled with which is low latency things so i love the examples with games that's actually like i mean i the doom example very good i have a benchmark on”

Claim: The speaker does not expect the model to solve everything, but believes it addresses a class of traditionally difficult low-latency problems.

“language models playing video games i've always been fascinated by whether you can have an intelligent system play new games and things of of this nature and so i think it's really really cool that they have”

Claim: The speaker is fascinated by whether intelligent systems can play unfamiliar video games.

“yeah yeah yeah um i guess these numbers are very outdated because a lot of the models are are very different and i've seen actually people run there there are some folks out there that are running newer models on these games which is really cool”

Claim: The speaker says the benchmark results are very outdated because models have changed substantially, and some people are testing newer models on the games.

“really cool i guess the general premise of this benchmark was we just want to see if vision language models are good enough at just like plugging into games with the latency constraint included because this this actually i came out with this right after clawed plays pokemon came out”

Claim: The benchmark was designed to evaluate whether vision-language models could play games under latency constraints.

“so this was like two years ago which i guess is like ancient now but i think what's really really cool about this suite of tasks it's very diverse in terms of what games they are and also i think most of the games are games that people know or like have seen before i saw yeah jeff playing doom i will say i don't think i think they were just playing like really simple levels and stuff”

Claim: The speaker says the benchmark was released about two years before the conversation.

“so this was like two years ago which i guess is like ancient now but i think what's really really cool about this suite of tasks it's very diverse in terms of what games they are and also i think most of the games are games that people know or like have seen before i saw yeah jeff playing doom i will say i don't think i think they were just playing like really simple levels and stuff”

Claim: The speaker describes the benchmark's game suite as diverse and composed mostly of familiar or previously seen games.

“so this was like two years ago which i guess is like ancient now but i think what's really really cool about this suite of tasks it's very diverse in terms of what games they are and also i think most of the games are games that people know or like have seen before i saw yeah jeff playing doom i will say i don't think i think they were just playing like really simple levels and stuff”

Claim: The game-task suite is diverse and consists mostly of games that people know or have seen before.

“but honestly like most models still can't really do or i don't actually think any models can solve these games very meaningfully like there are some games that they can i think i've seen astra be able to solve the kirby game and we also for this benchmark we intentionally designed a really minimal harness and i will get to this point about harnesses because i think there's a whole conversation to be had about like what is the value of a harness like what is even the”

Claim: The speaker believes most models—and possibly all models—still cannot meaningfully solve the games, while allowing that models can solve some individual games.

“but honestly like most models still can't really do or i don't actually think any models can solve these games very meaningfully like there are some games that they can i think i've seen astra be able to solve the kirby game and we also for this benchmark we intentionally designed a really minimal harness and i will get to this point about harnesses because i think there's a whole conversation to be had about like what is the value of a harness like what is even the”

Claim: The speaker has seen Astra solve the Kirby game.

“but honestly like most models still can't really do or i don't actually think any models can solve these games very meaningfully like there are some games that they can i think i've seen astra be able to solve the kirby game and we also for this benchmark we intentionally designed a really minimal harness and i will get to this point about harnesses because i think there's a whole conversation to be had about like what is the value of a harness like what is even the”

Claim: The team intentionally designed a minimal harness for the benchmark.

“but honestly like most models still can't really do or i don't actually think any models can solve these games very meaningfully like there are some games that they can i think i've seen astra be able to solve the kirby game and we also for this benchmark we intentionally designed a really minimal harness and i will get to this point about harnesses because i think there's a whole conversation to be had about like what is the value of a harness like what is even the”

Claim: The speaker believes current models generally cannot solve the benchmark games meaningfully, while noting that Astra has been able to solve the Kirby game.

“given the purpose of harness for a model but in general like i think it's uh yeah i i hope to see very quickly or very soon like all of these games be in by newer models”

Claim: The speaker hopes newer models will be able to play all of the games in the suite very soon.

“given the purpose of harness for a model but in general like i think it's uh yeah i i hope to see very quickly or very soon like all of these games be in by newer models”

Claim: The speaker hopes newer models will soon be able to play every game in the benchmark.

“i have been a little unsatisfied maybe with with how people think about harnesses because people compare like oh like i love cloud code i love codex i love pi like no i love oh my pi i love prime agent to be honest i think all of them are the same most of the design decisions or like design choices around these harnesses are the same maybe prime agent is a little bit different because it's like inherently an rlm”

Claim: The speaker thinks most design choices in Cloud Code, Codex, Pi, and Prime Agent harnesses are the same, although Prime Agent may differ somewhat because it is inherently an RLM.

“But in general, like, I think we can be a lot more creative with harnesses. And what I mean by that is, if we think about this from the perspective of what exactly is the harness doing for the model?”

Claim: The speaker thinks harness design can be much more creative than it currently is.

“Basically, when you're trying to solve a problem and you want to use a language model to solve it, like a very difficult task, one thing that we have discovered is that next token prediction is a really, really awkward form to do a lot of these tasks.”

Claim: The speaker says next-token prediction is an awkward format for using language models to perform many difficult tasks.

“like you just say solve code or like solve my query over this code base no so we rely on a harness to help you do these kinds of things and i think what what is interesting is like a harness is a very, very opinionated program over how you want a language model to be form fit over a problem.”

Claim: A harness is a program that encodes strong choices about how a language model should be adapted to a problem.

“We are now training models around harness rollouts and so there is this very awkward way of doing training over harnesses which is that we train a language model to act within a harness but like now it's like a really long maybe like multiple agent rollout that we're doing and there's like really awkward hacky ways of doing this.”

Claim: The speaker says current training around harness rollouts uses awkward, hacky methods to train language models to act within long, potentially multi-agent rollouts.

“the idea for this blog came with the RLM idea as well. We just didn't package it that way.”

Claim: The speaker says the blog idea originated alongside the RLM idea, although they did not present it as an RLM idea.

“There are many other ideas around RLMs that we will be coming out with, but we're all there from the beginning.”

Claim: The speaker says they plan to publish many additional ideas about RLMs.

“I think what I like about the RLM is that there were many, many, many iterations and versions of different abstractions that I was interested in doing. And ultimately the RLM made the most sense, but there's a lot of reasons that aren't public as to why that's the case.”

Claim: After considering many iterations and versions of different abstractions, the speaker concluded that the RLM made the most sense.

“And ultimately the RLM made the most sense, but there's a lot of reasons that aren't public as to why that's the case.”

Claim: The speaker says some reasons for choosing the RLM are not public.

“one of the things that we see with an RLM is that if you sufficiently offload context and ask the model to write code over that context, you get this really, really weird, but useful property, which is that when the model recognizes during training how to solve a task, it turns out that the solution to many tasks is”

Claim: The speaker says that in an RLM, sufficiently offloading context and having the model write code over it can reveal that solutions to many tasks are similar when the model learns how to solve them during training.

“is very similar across like tasks where you don't even like it's not even that clear to you that the solutions are similar so in this example we have like a retrieval task and we have like a an aggregation task and they're very very different query like the domain is just completely different and when you train a regular language model over these two tasks the trajectories look very different and so what you're relying when you like let's say you use pi or”

Claim: The speaker says retrieval and aggregation tasks may differ greatly in domain and in the trajectories of a regular language model, while sharing similar solutions.

“codcode or something which is not in this blog but we do have these results you'll find that like these harnesses distinguish too much between these problems even though the solutions are the same and so one thing that we find when training rlm's is that like when it sees these problems it's the same and the reason it sees these problems as the same is the sub agent sees different problems but the sub agent is solving an easier sub task and so you're”

Claim: The speaker says some harnesses, including Pi and Codex, treat problems as more distinct than warranted even when the problems share the same solution strategy.

“codcode or something which is not in this blog but we do have these results you'll find that like these harnesses distinguish too much between these problems even though the solutions are the same and so one thing that we find when training rlm's is that like when it sees these problems it's the same and the reason it sees these problems as the same is the sub agent sees different problems but the sub agent is solving an easier sub task and so you're”

Claim: The speaker says RLM training can make different problems appear equivalent at the overall-strategy level because a sub-agent handles an easier subtask for each problem.

“they end up looking the same. And so when you train on the left task, for example, it can immediately solve the right task.”

Claim: The speaker says an RLM trained on one task can immediately solve another task if the tasks share the same overall strategy.

“As you just naively train your RLM on these tasks, they naturally learn to generalize, for example, to longer tasks because the strategy is basically the same. You're just modifying like a length variable.”

Claim: The speaker says RLMs trained on tasks naturally generalize to longer versions when the underlying strategy remains the same and only the length changes.

“And this actually also holds for tasks that are different and they're not even they're not different across the length. They're completely different tasks, math tasks versus writing tasks. But the solution, the like meta high level solution is the same.”

Claim: The speaker says math and writing tasks can be very different while sharing the same high-level solution strategy.

“same and so when you train the rlm on one of them it generalizes this behavior to the second one and there's no magic here i guess maybe that's the thing that i want to kind of stress like”

Claim: The speaker says training an RLM on either a math task or a writing task can generalize the shared behavior to the other task, and that this generalization involves no magic.

“yeah they're learning how to solve these types of problems at a certain length and it turns out that when you take this strategy that they learned it is directly transferable to the longer length like they're effectively the same program and this is something that is really exciting because what this kind of implies is that when you have a corpus of data or environments that you train your model on the hope is that like”

Claim: The speaker says models trained to solve short tasks can transfer the learned strategy to tasks eight to thirty times longer when the longer tasks are effectively the same program.

“A, you can train on less environments and generalize some more than what existing models can do through harnesses through existing kind of like naive training.”

Claim: The speaker hopes that training on fewer environments will enable models to generalize more than existing harness-based, relatively naive training approaches.

“you still want to use all the data that you have.”

Claim: The speaker argues that training on selected tasks should not mean discarding available data; all available data should still be used.

“So when you train on these tasks, like hopefully it generalizes to a wider class of problems.”

Claim: The speaker hopes that training on these tasks will generalize to a wider class of problems.

“And why this is also even more exciting, at least in the context of RLMs or any recursively calling system, is this argument holds inductively.”

Claim: The speaker says the argument for generalization applies inductively to recursive language models (RLMs) and other recursively calling systems.

“So like, I'll give you an example because I said I claimed that competitive programming and GPU optimization use very similar skill sets.”

Claim: The speaker claims that competitive programming and GPU optimization rely on very similar skill sets.

“And so what I'm trying to get at with this whole blog post is just that like, we should rethink what the role of the harness is because harnesses can actually greatly increase the generalization capability of your model and the amount of data that it's given and this is not exclusive to RLMs I think there is a wide class of harnesses that are yet to be discovered that actually can yield similar properties and to extend this argument a little bit further I think what you can also extrapolate from this is like if I look at an RLM”

Claim: The speaker argues that harnesses can substantially improve a model's generalization capability and expand the amount of data it can use.

“what the role of the harness is because harnesses can actually greatly increase the generalization capability of your model and the amount of data that it's given and this is not exclusive to RLMs I think there is a wide class of harnesses that are yet to be discovered that actually can yield similar properties and to extend this argument a little bit further I think what you can also extrapolate from this is like if I look at an RLM”

Claim: The speaker believes that many kinds of harnesses beyond RLMs remain to be discovered and may provide similar generalization benefits.

“what are the components of an RLM that are actually necessary? And can I actually just directly train a model to do this? Can I train a model to act as an RLM implicitly in its forward pass?”

Claim: The speaker raises the possibility of training a model to behave like an RLM implicitly within its forward pass.

“you know, you might say like, oh, code is non-differentiable, blah, blah, blah, blah, blah. But there are many approximations of this behavior that we will start to uncover.”

Claim: The speaker expects that researchers will uncover many approximations of RLM-like behavior, despite the challenge of non-differentiable code.

“we will see beyond just like i'm going to design a new coding harness that uses a special form of compaction or something i think we can be a lot more creative here like there's there's so much we can do with these language models that i think we are just not doing and i'm like very excited about this because i think like i think we can get serious gains from very very opinionated and good harness design”

Claim: The speaker believes current harness design underuses language models and expects serious gains from creative, strongly opinionated harness designs.

“You know, the RLM, for example, is a very primitive inductive bias, like there's nothing super special about the design other than the fact that it's very different than what we currently do.”

Claim: The speaker characterizes the RLM as a primitive inductive bias whose design is not especially special except that it differs from current approaches.

“But this may potentially scale much better with like the data and the environments that we have available to us.”

Claim: The speaker suggests that the RLM approach may scale substantially better as available data and environments increase.

“yeah like most harness choices don't matter because all of these harnesses are the same but i will say like grock bot for example is actually quite different i think from my understanding than than how some of these other harnesses have been designed and i like that a lot um and i think it's clear from here at least that like i'm pretty sure anthropic or open ai are exclusively training on their harnesses they're probably not training on their competitors harness i mean i'd assume not because i don't know why they would do that but”

Claim: The speaker suspects Anthropic and OpenAI train only on their own harnesses and probably exclude competitors' harnesses from training.

“yeah like most harness choices don't matter because all of these harnesses are the same but i will say like grock bot for example is actually quite different i think from my understanding than than how some of these other harnesses have been designed and i like that a lot um and i think it's clear from here at least that like i'm pretty sure anthropic or open ai are exclusively training”

Claim: The speaker believes most harness choices make little difference because many harnesses are similar, while GrokBot differs substantially from some other harness designs.

“i think though as models get smarter or like as they get better this distinction becomes uh not that important that important in the sense that like if you take astra and you put it inside of open code like it's not gonna go crazy because i think it's like just sufficiently good and so i say this because the only benefits between these different harnesses is just cost for the most part and and i think like what i'm getting at here is if you plug these models into rlm's though”

Claim: The speaker thinks that as models improve, differences between harnesses matter less, and that Astra is capable enough to work in OpenCode without behaving badly.

“that important in the sense that like if you take astra and you put it inside of open code like it's not gonna go crazy because i think it's like just sufficiently good and so i say this because the only benefits between these different harnesses is just cost for the most part and and i think like what i'm getting at here is if you plug these models into rlm's though”

Claim: The speaker thinks that differences in cost account for most of the benefits of choosing among harnesses.

“they're not that good still they're okay and i think it's mainly because the types like the class of harness that we are training around is this class of harness this like pie loop this like i like to call it trajectory as a prompt which just means like you keep the whole trajectory of the rollout as the context that that your main model is is using even if you use sub-agents it's still like kind of this form”

Claim: The speaker thinks models used in RLMs remain only moderately capable, mainly because training has focused on harnesses that keep the rollout trajectory in the model's context as a prompt.

“And I think we're going to, if we want to explore new harnesses, there needs to be teams that are dedicated to. actually running meaningful experiments over like scaling out new harnesses like maybe post-train scaling out on different harness designs like i think we can actually get very meaningful knowledge or gains from doing this kind of thing whether it's an rlm or whether it's something different and that's exciting because i think for example if you train a lot on fable for a long time was the best model for rlm because they had dynamic workflows and it was”

Claim: The speaker argues that dedicated teams should experimentally scale new harness designs, including through post-training, because doing so could yield meaningful knowledge or capability gains.

“actually running meaningful experiments over like scaling out new harnesses like maybe post-train scaling out on different harness designs like i think we can actually get very meaningful knowledge or gains from doing this kind of thing whether it's an rlm or whether it's something different and that's exciting because i think for example if you train a lot on fable for a long time was the best model for rlm because they had dynamic workflows and it was And it was pretty obvious that like this was a capability that was somewhat trained in,”

Claim: The speaker says Fable was the best model for RLM for a long time because it supported dynamic workflows, suggesting that this capability had been trained into it to some extent.

“even if the model was still like a little dumb, like in the RLM harness, it still worked a lot better than than other models did.”

Claim: The speaker says Fable performed much better than other models in the RLM harness, even when Fable itself was still somewhat unintelligent.

“Astra is now also like good enough at doing these things.”

Claim: The speaker says Astra is now capable enough to perform the relevant RLM-related tasks.

“these are all internal results, I don't I don't have them public right now. But in general,”

Claim: The speaker says the results under discussion are internal and not publicly available at the time of the conversation.

“You can very easily tell that we have not optimized for RLM like workflows yet.”

Claim: The speaker says their models or systems have not yet been optimized for RLM-like workflows, and that this is readily apparent.

“models that do this correctly they will be a lot more efficient as well i mean you can kind of just think through like why this is the case right oh”

Claim: The speaker predicts that models that correctly perform RLM-like workflows will be substantially more efficient.

“An RLM is basically just a harness design where the only tool in the harness is code,”

Claim: An RLM is a harness design whose sole tool is code.

“which is this programmatic sub-agent calling thing where it has the option to call itself as a tool and it has other tools. But all of these things are functions in code and the context that it's dealing with is all.”

Claim: An RLM can let a model call itself and other tools through functions represented in code.

“always stored in some memory inside of this code environment so this could be a file system like this could be like i'll give an example prime agent the trajectory of prime agent like the context even the when you compact and do all these things is stored on disk so the model can always reference its original context even if it's compacted and all of its tools are run inside of let's say like a python repl or a bash repl”

Claim: RLMs store working context in memory within a code environment; in the Prime Agent example, the trajectory and context remain on disk and can be referenced after compaction.

“trouble and so this it's like this very primitive abstraction and i would say like where most harnesses differ is a context offloading is not done that like that i mean if you look at prime agent by the way prime agent does context offloading but not all the way so like it still maintains the standard codex loop of like trajectory as a prompt where you compact but it has the additional kind of like the context is offloaded and it only has the unique point of of prime agent is that the only tool is is ipython”

Claim: Harnesses vary in context offloading. Prime Agent offloads context partially while retaining the Codex trajectory-as-a-prompt loop, and its distinctive feature is using IPython as its sole tool.

“Yeah. So the original problem was harnesses have a really bad time dealing with long context.”

Claim: RLMs originally addressed the difficulty that harnesses have handling long contexts.

“They typically used to only really do them for like specific things like code. like you know they could deal with your code base because it was trained on it but now it's more around what this blog is talking about which is compositionality and the fact that like i think harnesses we want to have language model systems that have much more control over the actions they make at every step”

Claim: The speaker says harnesses used to be suited mainly to specific tasks such as code, where models had been trained on the relevant codebase.

“like you know they could deal with your code base because it was trained on it but now it's more around what this blog is talking about which is compositionality and the fact that like i think harnesses we want to have language model systems that have much more control over the actions they make at every step”

Claim: The speaker argues that compositional language-model systems should have substantially more control over their actions at each step.

“And what I mean by this is tool calls are very limited because you have to invoke them every turn. Like you have to invoke tool A, then tool B, then tool C. And there's no central context that you can kind of draw back from.”

Claim: The speaker says conventional tool-call workflows are limited because tools must be invoked sequentially across turns and lack a shared central context.

“And RLMs are specifically like designed around composition and having like a central context that you can always draw from.”

Claim: RLMs are designed to support composition through a central context that remains available for reference.

“Like these agent swarms have like a message board that they that they learn to communicate over and this message board in some sense.”

Claim: In the agent-swarm example, agents learn to communicate through a shared message board that serves as their common context.

“since is the shared context that they like act over and rlm's basically say that like the best way to communicate through this is in code like you write the code to do this and it's it's because these models are so good at writing code um like we want to take advantage of of that fact exactly”

Claim: RLMs use code as the medium for working with shared context, taking advantage of models' strong code-writing ability.

“If an RLM breaks down its computation into like a kind of a meta harness of sorts or like a program that involves sub agents that like look at a local problem, every individual language model call over the course of this task is in distribution, even if the entire task itself is out of distribution.”

Claim: The speaker expects compositional harnesses to make each language-model call in distribution, even when the overall task is out of distribution.

“And this is a very desirable property, I think, for obvious reasons, like if every task is in distribution for each individual language. language model call you will probably get to the right answer um so exactly”

Claim: The speaker considers in-distribution individual language-model calls desirable because they will probably produce the right answer.

“I am not working on this at MIT, or at least in the scaled sense, because I can't afford to.”

Claim: The speaker is not working on RLMs at MIT at scale because he cannot afford to.

“But there are companies out there that are working on this. I think Prime and Select is very clearly working on this.”

Claim: The speaker thinks Prime and Select is clearly working on RLMs or related harness designs.

“I'm very excited to see maybe we'll observe, I don't know, but maybe we'll observe better post-training scaling laws with when you train around a smart harness.”

Claim: The speaker is excited to see whether training around a smart harness will yield better post-training scaling laws.

“Maybe we'll even see smarter harnesses. that come out and like they they work better around these kinds of principles no”

Claim: The speaker thinks harnesses may emerge that work better according to the discussed compositional principles.

“what more of what i mean is like cloud code codex pi etc are all the same in that like when you break down the the logic of the harness it's like virtually the same thing”

Claim: The speaker says Claude Code, Codex, and Pi use harness logic that is virtually the same.

“with orlms and and with other harness abstractions it looks very different and this is where i think you really distinguish i mean it's it's in the same way that like i think with language model architecture choices a lot of architecture choices end up kind of looking the same when you like scale it out or like the differences end up being like somewhat minor in terms of i mean for a lab it's not minor but you know maybe one model converges better than the other one like slightly but in general like if you were so for example pre-training scaling laws only hold because the architecture choices we have are somewhat stable right now but if you were to completely change the architecture pre-training scaling laws probably don't hold or like these kinds of this like power law is going to look very different and it's like the same thing with harnesses like i think all the harnesses we have right now for the most part are roughly look the same, but there are some exceptions to this, I think, that are coming out.”

Claim: The speaker says ORLMs and other harness abstractions differ substantially from most current harnesses, which generally look similar.

“you know maybe one model converges better than the other one like slightly but in general like if you were so for example pre-training scaling laws only hold because the architecture choices we have are somewhat stable right now but if you were to completely change the architecture pre-training scaling laws probably don't hold or like these kinds of this like power law is going to look very different and it's like the same thing with harnesses like i think all the harnesses we have right now for the most part”

Claim: The speaker says pre-training scaling laws depend on current architecture choices remaining relatively stable; a complete architecture change would probably invalidate those laws or substantially alter the power-law relationship.

“but there are some exceptions to this, I think, that are coming out.”

Claim: The speaker says some harnesses that differ from the broadly similar current designs are emerging.

“involved with prime uh after they they released a blog post by the way not affiliated with me at all about like how they believed rlm's were kind of the the future and i i had a friend that was working there gpu mode mate like we got in touch and i think i agreed with a lot of the researchers there and like what they believed about harness design like i i was i was very impressed i think that like they understood”

Claim: The speaker agreed with many Prime researchers about harness design and was impressed by their understanding of it.

“involved with prime uh after they they released a blog post by the way not affiliated with me at all about like how they believed rlm's were kind of the the future and i i had a friend that was working there gpu mode mate like we got in touch and i think i agreed with a lot of the researchers there and like what they believed about harness design like i i was i was very impressed i think that like they understood”

Claim: The speaker says he became involved with Prime after its blog post argued that RLMs were the future and he connected with a friend who worked there.

“the purpose of the arlem paper which is not necessarily just to say that like you know we're solving long context tasks but actually like we want more opinionated harness designs yeah yes”

Claim: The speaker says the RLM paper aimed not merely to solve long-context tasks but to encourage more opinionated harness designs.

“after after we decided to work together we decided to look into training an rlm and also build this kind of rlm harness and kind of see where we can take it that is how like prime agent came about and i think the reception for prime agent has been pretty good like the one thing i was worried about with prime agent is that none of them at least at the time when we were building it none of the models were that good at doing rlm stuff so this was like pre-fable”

Claim: The speaker says Prime Agent arose from a collaboration to investigate training an RLM and building an RLM harness.

“after after we decided to work together we decided to look into training an rlm and also build this kind of rlm harness and kind of see where we can take it that is how like prime agent came about and i think the reception for prime agent has been pretty good like the one thing i was worried about with prime agent is that none of them at least at the time when we were building it none of the models were that good at doing rlm stuff so this was like pre-fable”

Claim: The speaker says Prime Agent received a pretty good reception.

“after after we decided to work together we decided to look into training an rlm and also build this kind of rlm harness and kind of see where we can take it that is how like prime agent came about and i think the reception for prime agent has been pretty good like the one thing i was worried about with prime agent is that none of them at least at the time when we were building it none of the models were that good at doing rlm stuff so this was like pre-fable pre-Astra um”

Claim: The speaker was concerned that the models available during Prime Agent's development were not very capable at RLM tasks.

“a harness on top of pi like pi mono which is pi mono for context is like the yeah like harness i use pi as the reference for everything because i think all other harnesses are basically just pi um but it is pi except we explicitly restrict ipython to be the only tool that's available to it every other tool gets loaded in as like a python module um or like a like a bash kind of script that it can”

Claim: Alex Zhang describes Prime Agent as a harness built on Pi and says he uses Pi as his reference for other harnesses, which he considers basically Pi-like.

“yeah like harness i use pi as the reference for everything because i think all other harnesses are basically just pi um but it is pi except we explicitly restrict ipython to be the only tool that's available to it every other tool gets loaded in as like a python module um or like a like a bash kind of script that it can”

Claim: Prime Agent makes IPython the model's only directly available tool, while exposing other tools through Python modules or runnable bash-like scripts.

“it can run so it uses the core rlm abstraction on top of pi and then it also has this continual harness thing which is uh seth he's another phd student this is a thing that he used to get language model harnesses to play games like so he worked a lot with joel uh who is the like gemini plays pokemon guy and continual harness is also by the way very simple i i quite like it”

Claim: Prime Agent combines the core RLM abstraction with Pi and includes a continual harness.

“It basically is this design principle around like what parts of the harness can you let the harness itself modify?”

Claim: The continual harness is designed around allowing the harness to modify selected parts of itself.

“There are certain pieces that like you'll let it modify its own skills, the sub agents available to it, what the system prompt to the model is,”

Claim: The continual harness can modify selected components, including its skills, available subagents, and the model's system prompt.

“and continual harness is available basically as a tool inside of the IPython kernel.”

Claim: The continual harness is exposed as a tool within the IPython kernel.

“standard right yeah i think what is what i i really liked about it and we got kind of lucky is that like a lot of the new frontier models actually work really well inside and actually even a lot of the open source models work really well at least some of the newer ones and there is another thing in prime agent i should highlight which is that like we have a very particular agent to agent”

Claim: Alex Zhang says many new frontier models and some newer open-source models work well within Prime Agent.

“agent communication system or like uh like framework which is because rlm's tend to spawn many sub agents we want a way for sub agents to communicate with maybe the root or with each other and so there are some design decisions around like what each sub agent is allowed to talk to how it does it again everything is in code so it writes the code to do this kind of communication which i think is really really cool and then there's i guess persistent sub agents is another”

Claim: Prime Agent provides an agent-to-agent communication framework for subagents to communicate with the root agent or with each other, addressing the tendency of RLMs to spawn many subagents.

“agent communication system or like uh like framework which is because rlm's tend to spawn many sub agents we want a way for sub agents to communicate with maybe the root or with each other and so there are some design decisions around like what each sub agent is allowed to talk to how it does it again everything is in code so it writes the code to do this kind of communication which i think is really really cool and then there's i guess persistent sub agents is another”

Claim: Prime Agent's subagent communication rules specify permitted communication partners and methods, and the communication is implemented in code written by the agents.

“which is the sub-agents, they can last beyond like the standard runtime of the actual like original agent and you can go into that sub-agent, you can prompt it more, like you have more visibility and flexibility into what is kind of going on.”

Claim: Prime Agent's persistent subagents can continue beyond the original agent's standard runtime and can be revisited, prompted further, and inspected by users.

“that is the big trick.”

Claim: Alex Zhang identifies externalizing state to a file system as the key technique for persistent agent behavior.

“yeah i mean they're they're showing that they can train it on their their kind of hosted training stack but no so for model training i'm i'm not involved with them on that the main reason is just i have other things in the phd i want to work on i think like there are many other big bets to take um outside of just rlms uh some uh some i i don't know how much i can share yet uh but in general like i think i actually think one of the luxuries of being”

Claim: Alex Zhang believes there are many significant research directions to pursue beyond RLMs.

“yeah i mean they're they're showing that they can train it on their their kind of hosted training stack but no so for model training i'm i'm not involved with them on that the main reason is just i have other things in the phd i want to work on i think like there are many other big bets to take um outside of just rlms uh some uh some i i don't know how much i can share yet uh but in general like i think i actually think one of the luxuries of being”

Claim: Alex Zhang is not involved in the other party's model training, primarily because he wants to focus on other work in his PhD.

“of being a phd student genuinely is that there's so many big bets to take most of them will probably yield nothing but it's a really really exciting time to be in research especially because i think most progress in the field has been a little bit boring like i'm not saying the the outcome is boring but the process of of doing these things tends to be quite boring um and so there is kind of this question of like you know what”

Claim: Alex Zhang considers the abundance of research bets a luxury of PhD study, while expecting that most of those bets will probably produce no results.

“of being a phd student genuinely is that there's so many big bets to take most of them will probably yield nothing but it's a really really exciting time to be in research especially because i think most progress in the field has been a little bit boring like i'm not saying the the outcome is boring but the process of of doing these things tends to be quite boring um and so there is kind of this question of like you know what”

Claim: Alex Zhang considers the current period especially exciting for research.

“of being a phd student genuinely is that there's so many big bets to take most of them will probably yield nothing but it's a really really exciting time to be in research especially because i think most progress in the field has been a little bit boring like i'm not saying the the outcome is boring but the process of of doing these things tends to be quite boring um and so there is kind of this question of like you know what”

Claim: Alex Zhang thinks many processes behind progress in the field have been somewhat boring, although their outcomes are not necessarily boring.

“So Harvey, the legal AI company, released a blog post not affiliated at all, but they post trained an RLM on their. they're like legal work which often involves a lot of like sifting through documents and kind of looking through like a variety of specific information that maybe is not so easy to retrieve with like a pure retrieval system and they show like really really good results it's very exciting i mean i i i was i was shocked that that they worked on this they did not tell me so when this came out i was like oh that's awesome so there's this one i think super super cool and what they're doing”

Claim: Harvey published a blog post describing an RLM trained for legal work that involves sifting through documents and finding information that may be hard to retrieve using a pure retrieval system.

“released a blog post not affiliated at all, they're like legal work which often involves a lot of like sifting through documents and kind of looking through like a variety of specific information that maybe is not so easy to retrieve with like a pure retrieval system and they show like really really good results it's very exciting i mean i i i was i was shocked that that they worked on this they did not tell me so when this came out i was like oh that's awesome so there's this one i think super super cool and what they're doing”

Claim: Alex Zhang says Harvey's RLM work was unaffiliated with him and that he learned of it only when it was released, because Harvey had not told him beforehand.

“they're like legal work which often involves a lot of like sifting through documents and kind of looking through like a variety of specific information that maybe is not so easy to retrieve with like a pure retrieval system and they show like really really good results it's very exciting i mean i i i was i was shocked that that they worked on this they did not tell me so when this came out i was like oh that's awesome so there's this one i think super super cool and what they're doing”

Claim: Alex Zhang describes the results reported for Harvey's legal RLM as very good and the work as exciting.

“i think this is a collaboration with base 10”

Claim: Alex Zhang thinks Harvey's legal RLM work may have been done in collaboration with Base10.

“headlong which is law the laud institute's kind of uh it is their like persistently running harness um it's very cool”

Claim: Headlong is a persistently running harness associated with the Law Institute.

“they are using the rlm abstraction but they're doing something much cooler than the rlm uh which is like they have a system that kind of what they call like thinks persistently so even when you don't query it it has a way to think through problems that it has in its context”

Claim: Alex Zhang says Headlong uses the RLM abstraction and can continue thinking about problems in its context even when it is not being queried; he considers this more interesting than the RLM abstraction itself.

“the official one not not the like public prime and select one that or like what people have evaluated on they all like claim to use or they reference like like tufa for example some form or some inspired form of the arlen abstraction in their harness which is really really cool i think it's um this is where i mean this is exactly the setting where you would see a lot of benefits from composition and and using code and combining like neurosymbolic systems with with ai and”

Claim: The speaker believes that harnesses in the official ARC-AGI-3 Kaggle competition referenced Tufa or an abstraction inspired by it.

“the official one not not the like public prime and select one that or like what people have evaluated on they all like claim to use or they reference like like tufa for example some form or some inspired form of the arlen abstraction in their harness which is really really cool i think it's um this is where i mean this is exactly the setting where you would see a lot of benefits from composition and and using code and combining like neurosymbolic systems with with ai and”

Claim: The speaker believes that ARC-AGI-3 is a setting where composing systems, using code, and combining neuro-symbolic systems with AI can provide substantial benefits.

“the official one not not the like public prime and select one that or like what people have evaluated on they all like claim to use or they reference like like tufa for example some form or some inspired form of the arlen abstraction in their harness which is really really cool i think it's um this is where i mean this is exactly the setting where you would see a lot of benefits from composition and and using code and combining like neurosymbolic”

Claim: The speaker says the official ARC-AGI-3 harnesses claimed to use or referenced Tufa or an abstraction inspired by it.

“i mean i i would guess probably not unless you say like i'll be i'll be careful here lawyer because you know people debate what is an rlm what is not an rlm it's somewhat clear that what they used is some kind of swarm of agents with a shared some shared context like some shared file system and like this is very much in the spirit of rlm stuff but i think there's a there's a lot of like you know more clever things that they did that's not maybe related to to the rlm itself i agree a little bit with the idea that like”

Claim: The speaker guesses that OpenAI probably did not use an RLM, while noting that what counts as an RLM is debated.

“lawyer because you know people debate what is an rlm what is not an rlm it's somewhat clear that what they used is some kind of swarm of agents with a shared some shared context like some shared file system and like this is very much in the spirit of rlm stuff but i think there's a there's a lot of like you know more clever things that they did that's not maybe related to to the rlm itself i agree a little bit with the idea that like”

Claim: The speaker says OpenAI's approach clearly involved a swarm of agents sharing context, possibly through a shared file system, and was in the spirit of RLMs.

“lawyer because you know people debate what is an rlm what is not an rlm it's somewhat clear that what they used is some kind of swarm of agents with a shared some shared context like some shared file system and like this is very much in the spirit of rlm stuff but i think there's a there's a lot of like you know more clever things that they did that's not maybe related to to the rlm itself i agree a little bit with the idea that like”

Claim: The speaker thinks some of the clever elements in OpenAI's approach may have been unrelated to the RLM itself.

“like a harness is not that necessary for what they did the way that i would put this is that i think a model like a gpc6 astra type thing is technically smart enough conditioned on the right information to come up with a proof for these very very difficult problems now how you get to that information is a giant question mark and in their case it probably came down to like a very long search over like many of these sub or many of these like agents in the swarm and maybe also like researchers can i'm actually not sure about this part but putting in like their kind of intuition as to like what you should explore and things like this and ultimately like this produced some information that some agent was able to take the take to finish the proof and so in that sense like i think was the harness that important no and i i think what this is pointing at is like these specific details of a harness do not really matter and i think that's also what that uh what the harness tactics paper is pointing at which is that like beyond the user's feeling of the harness”

Claim: The speaker thinks a harness was not very necessary for OpenAI's approach.

“like a harness is not that necessary for what they did the way that i would put this is that i think a model like a gpc6 astra type thing is technically smart enough conditioned on the right information to come up with a proof for these very very difficult problems now how you get to that”

Claim: The speaker believes a model like GPT-6 Astra is technically capable of producing proofs for very difficult problems if it is given the right information.

“information is a giant question mark and in their case it probably came down to like a very long search over like many of these sub or many of these like agents in the swarm and maybe also like researchers can i'm actually not sure about this part but putting in like their kind of intuition as to like what you should explore and things like this and ultimately like this produced some information that some agent was able to take”

Claim: The speaker says it remains highly uncertain how to provide a model with the information needed to prove very difficult problems.

“information is a giant question mark and in their case it probably came down to like a very long search over like many of these sub or many of these like agents in the swarm and maybe also like researchers can i'm actually not sure about this part but putting in like their kind of intuition as to like what you should explore and things like this and ultimately like this produced some information that some agent was able to take”

Claim: The speaker thinks OpenAI's approach probably used a long search across many agents and may have incorporated researchers' intuition about what to explore, though the speaker is unsure whether researchers contributed that intuition.

“the take to finish the proof and so in that sense like i think was the harness that important no and i i think what this is pointing at is like these specific details of a harness do not really matter and i think that's also what that uh what the harness tactics paper is pointing at which is that like beyond the user's feeling of the harness realistically all that matters is just like how are you composing these agents in a meaningful way to get to the final answer and maybe that's what like swarms and all these things are are really about and so from my pov at least if we start to think about like for user use cases what do we want out of harnesses and and things like this like”

Claim: The speaker believes that specific harness details matter less than meaningfully composing agents to produce a final answer.

“we want to take the good parts out of these like the cod codes the codecs is like the stream that people like to see but like under the hood whatever is running can be some really weird complicated swarm of agents that like ultimately come up with an answer the user doesn't want to see that though obviously right like it's it's not legible information and so i you know this was another kind of thing in the spirit of rlms like recursive language model it sounds like it's a language model and but it's not It's not a language model architecture.”

Claim: An RLM is not a language-model architecture, according to the speaker.

“we want to take the good parts out of these like the cod codes the codecs is like the stream that people like to see but like under the hood whatever is running can be some really weird complicated swarm of agents that like ultimately come up with an answer the user doesn't want to see that though obviously right like it's it's not legible information and so i you know this was another kind of thing in the spirit of rlms like recursive language model it sounds like it's a language model and but it's not”

Claim: The speaker believes users generally should not have to see the complicated, potentially illegible swarm of agents operating behind a user-facing interface.

“But the reason for this is like, I think we will start to see in the future probably one day and I wrote a blog about this, what we think of as a language model, like the thing that we query might actually be like a swarm or like a scaffold or like some. weird harness design that scales very well but the user just doesn't see it ultimately all the user sees is some front end version of this harness and yeah i i i think it's a relatively safe bet at least to make that this is what we will see”

Claim: The speaker predicts that users may eventually query a well-scaling swarm, scaffold, or harness as if it were a language model, while seeing only a front-end interface.

“um and this maybe goes back to the limitations of the base transformer like you know obviously if you just took a base transformer and you said like solve navier stokes or something it's not going to do it like yeah we all know this is not what's”

Claim: A base transformer would not solve the Navier–Stokes problem merely by being prompted to solve it, according to the speaker.

“not what's going to happen but yeah i think this is maybe the more interesting part uh and maybe the claims around like you know did the harness matter is more around this of like just arbitrarily pointing models like or agents at a growing kind of context of information maybe it's just enough to solve very difficult problems um that i can buy”

Claim: The speaker thinks that directing models or agents at an expanding context of information might be sufficient to solve very difficult problems.

“this is a bottleneck if there's one main guy uh it's coordinating all the sub guys then they have to like you know gather again and then re-coordinate that's slow that's that's crappy uh what a true swarm should be is everyone is just their own person right”

Claim: A centralized agent swarm where one main agent coordinates subagents and gathers their results is slow and creates a bottleneck.

“this is a bottleneck if there's one main guy uh it's coordinating all the sub guys then they have to like you know gather again and then re-coordinate that's slow that's that's crappy uh what a true swarm should be is everyone is just their own person right”

Claim: A true agent swarm should consist of agents acting independently rather than being centrally coordinated.

“And there are many settings where an RLM can solve maybe a more difficult task than compaction can,”

Claim: In many settings, an RLM may solve tasks that are more difficult than those solvable by compaction.

“but you would prefer to use compaction in most cases because it's cheaper and it's quicker.”

Claim: Compaction is preferable to an RLM in most cases because it is cheaper and quicker.

“And I think in the context of agent swarms, there is a similar thing going on of like, I'm fairly certain that like 95% of the swarm is entirely useless or like what it's exploring is entirely.”

Claim: The speaker is fairly certain that about 95% of an agent swarm, or of what it explores, is entirely useless.

“you're just burning tokens versus in this setup maybe not so much i'm not sure i mean maybe it's also the case”

Claim: An agent swarm with largely useless exploration wastes tokens.

“you're just burning tokens versus in this setup maybe not so much i'm not sure i mean maybe it's also the case”

Claim: The speaker is unsure whether the token-waste concern applies to the same extent in the setup under discussion.

“i think at some level that's how problems are framed right so like if you have a search problem and you're spanning out a bunch of sub agents to do search there's going to be a lot of useless information right there is one retrieval answer that you're getting and you're spanning off but that is consciously understood right”

Claim: When many subagents are deployed for a search problem, they produce substantial useless information, which is consciously accepted in that problem framing.

“yeah maybe maybe you'll have to pay like 40 million dollars to get a result and it's like well but it's exciting i i will say like it is it's very exciting that we even have the option to point 40 million dollars at a problem and solve it but there is there is still kind of a lot of research to be done in this area around around like you know what is necessary like what do we want to do what design do we want i mean we probably don't want everything to be a swarm but like where do we draw the line like can the agent design that or decide that for itself etc etc so”

Claim: More research is needed to determine which agent-swarm designs are appropriate and when they should be used.

“yeah maybe maybe you'll have to pay like 40 million dollars to get a result and it's like well but it's exciting i i will say like it is it's very exciting that we even have the option to point 40 million dollars at a problem and solve it but there is there is still kind of a lot of research to be done in this area around”

Claim: The speaker finds it exciting that it is possible to spend $40 million to solve a problem.

“around like you know what is necessary like what do we want to do what design do we want i mean we probably don't want everything to be a swarm but like where do we draw the line like can the agent design that or decide that for itself etc etc so”

Claim: The speaker is unsure whether an agent can decide for itself when an agent swarm should be used.

“around like you know what is necessary like what do we want to do what design do we want i mean we probably don't want everything to be a swarm but like where do we draw the line like can the agent design that or decide that for itself etc etc so”

Claim: The speaker thinks agent swarms probably should not be used for every problem.

“i think with open-endedness problems like i view them as somewhat similar to even like unsolved math problems maybe that's a weird way of of framing it but like i think a lot of the techniques in terms of like how people like approach them are kind of the same like evolutionary search is like very similar to just launching swarms of agents and hoping that like they come up with like an interesting and this is what”

Claim: The speaker views open-endedness problems as somewhat similar to unsolved mathematical problems.

“i think with open-endedness problems like i view them as somewhat similar to even like unsolved math problems maybe that's a weird way of of framing it but like i think a lot of the techniques in terms of like how people like approach them are kind of the same like evolutionary search is like very similar to just launching swarms of agents and hoping that like they come up with like an interesting and this is what”

Claim: The speaker thinks approaches to open-endedness problems and unsolved mathematical problems often share techniques, including evolutionary search and launching agent swarms to find interesting results.

“is what what like alpha evolved and some of these other works did uh like a year or two ago but i think what maybe is is not uh and i'm not sure if this is what you were alluding to but i think what's not super clear in open-endedness style search problems is do we frame all of them as basically like an unsolved very difficult problem or like where the objective is clear you”

Claim: AlphaEvolve and some other works used evolutionary-search or agent-swarm-like approaches about one or two years before the conversation.

“is what what like alpha evolved and some of these other works did uh like a year or two ago but i think what maybe is is not uh and i'm not sure if this is what you were alluding to but i think what's not super clear in open-endedness style search problems is do we frame all of them as basically like an unsolved very difficult problem or like where the objective is clear you clear if that's not the case i still don't know yet entirely what the value of it is maybe you have other opinions like i i don't have too many opinions on this but at least from my time it's kind of like i i got the sense that like”

Claim: The speaker is unsure whether every open-ended search problem should be framed as a very difficult unsolved problem with a clear objective.

“clear if that's not the case i still don't know yet entirely what the value of it is maybe you have other opinions like i i don't have too many opinions on this but at least from my time it's kind of like i i got the sense that like”

Claim: The speaker does not yet fully understand the value of open-endedness in cases where the objective is unclear.

“we ultimately still kind of want to approach things the way that like say open ai approach snubby or stokes we want we there's still a lot of nudging in certain directions that we want to have to like get to the point where we have something interesting”

Claim: Based on the speaker's experience, open-ended search still requires some directional nudging to produce interesting results.

“the biggest issue that like maybe there's a clear path to this but when i was there at least the biggest issue that people had with open-endedness was like how do you ultimately choose at the end how do you pick out the interesting stuff because when there is no like maybe the agent comes up with a”

Claim: The speaker says that, at the time they were there, the biggest concern about open-endedness was how to select the interesting results from the output.

“with a goal but in a lot of cases like what they and they have something called fugu i think which is like a it's like a model router type thing that was like inspired at least by this idea of like let's pick a problem where maybe we can pick out the best solutions to something in this case it's like pick the best model for this problem um”

Claim: The speaker thinks Fugu is a model-router-type system inspired by selecting the best model for a given problem.

“no no yeah but so i i bring this up because i think like with open-endedness like just generally the issue is like is like when we have this giant corpus of like slop like how do we sift through and”

Claim: A general challenge in open-endedness is filtering a large corpus of low-quality output.

“find like the hidden gems and like the solution to open-endedness really just letting models run forever and like finding like just doing data gen just doing super super super high throughput data generation and then like asking agents to go through and like and like fine meaningful things like i'm not sure maybe maybe that's sufficient you know like that would be that'd be cool”

Claim: The speaker suggests that open-endedness might be addressed by running models indefinitely, generating data at very high throughput, and having agents search for meaningful findings, but is unsure whether this approach is sufficient.

“yeah yeah yeah um i mean i actually love the people there like i i think they have a really really smart team uh and it makes sense i mean it branched off from from like an earlier team at gdm which was also kind of i guess doing this kind of like like open-ended evolutionary research style stuff what i liked about my experience there at least was that they did have that like mishap back i forget at this point when but i think”

Claim: The speaker considers Sakana's team very smart and says Sakana branched off from an earlier GDM team working on open-ended evolutionary research.

“yeah and i guess like the ai scientists like i mean there's there's some criticism of it that i mean i don't i don't work on that so i have no no kind of take on it but”

Claim: The speaker has no particular view on criticism of Sakana's AI scientists because they do not work on that area.

“but i think in general like what i like about them at least is that they're a little bit more of a researchy type lab so like they don't operate in the same space as like open ai or anthropic like for sure like”

Claim: The speaker likes that Sakana is more research-oriented and operates in a different space from OpenAI and Anthropic.

“no they do not i mean it's pretty obvious probably that at least when i was there they do not have a big competitor model or something that like you know that everyone is is using but i think they kind of operate in some ways as like a”

Claim: At the time the speaker was at Sakana, Sakana did not have a major competitor model that many people used.

“like a phd lab which is cool like and i i i think like david ha is like he's he's really really smart like i think he has a good sense of like also i think the market in japan is also a little bit different for ai and like who they're targeting is slightly different than maybe what we're used to here but yeah i like that they take kind of a lot of their research is kind of weird i think when people view it you know and i like that like i think it's”

Claim: The speaker believes Japan's AI market and Sakana's target audience differ somewhat from the market and audience the speaker is used to.

“like a phd lab which is cool like and i i i think like david ha is like he's he's really really smart like i think he has a good sense of like also i think the market in japan is also a little bit different for ai and like who they're targeting is slightly different than maybe what we're used to here but yeah i like that they take kind of a lot of their research is kind of weird i think when people view it you know and i like that like i think it's”

Claim: The speaker says Sakana operates in some ways like a PhD lab and appreciates that much of its research is unusual.

“Like the way it works there is a bit different, like how deals happen and stuff like that.”

Claim: The speaker says business practices in Japan, including how deals are made, differ somewhat from those elsewhere.

“yeah yeah well he's he's super cracked at least my mind understand i mean i i think in general like a lot of a lot of the chinese labs have done really really cool work”

Claim: The speaker believes many Chinese AI labs have produced impressive work.

“um one thing i will say is whatever open ai is doing with their agent swarm is like clearly the right thing to do you have to kind of think about it this way like nothing especially without like a very very smart harness design which i i don't i don't think anyone really has so far”

Claim: The speaker considers OpenAI’s agent-swarm approach the right direction, while saying that effective agent swarms require a very smart harness design that, in the speaker’s view, nobody has yet achieved.

“the agent swarm design is not something you can just take for granted like it's not like gpt6 astra is just super smart and then it just got agent swarms running well they clearly train i mean the the hugging face incident was them training a system to be like a swarm uh and i think like clearly they've done something really well to the point where you can throw 40 million dollars and solve an unsolved problem”

Claim: The speaker describes the Hugging Face incident as involving the training of a system to operate as a swarm.

“the agent swarm design is not something you can just take for granted like it's not like gpt6 astra is just super smart and then it just got agent swarms running well they clearly train i mean the the hugging face incident was them training a system to be like a swarm uh and i think like clearly they've done something really well to the point where you can throw 40 million dollars and solve an unsolved problem”

Claim: The speaker believes OpenAI’s work on an agent swarm was good enough to enable it to solve an unsolved problem, with $40 million being spent on the effort.

“the agent swarm design is not something you can just take for granted like it's not like gpt6 astra is just super smart and then it just got agent swarms running well they clearly train i mean the the hugging face incident was them training a system to be like a swarm uh and i think like clearly they've done something really well to the point where you can throw 40 million dollars and solve an unsolved problem”

Claim: The speaker says effective agent-swarm design does not arise automatically from a highly capable model such as GPT-6 Astra; it requires deliberate work.

“solve problem and i i think with the kimmy agent swarm thing like at least from when i read it it just came off as like this is interesting but i don't actually know whether or not this can solve anything novel they”

Claim: Based on what the speaker had read, the speaker was unsure whether the Kimi agent swarm could solve any novel problems.

“and i i will say the same thing with dynamic workflows i actually kind of think the dynamic workflows release was sort of a flop i don't know how you guys feel about it but i think it's like uh my understanding is like it's not used that often or”

Claim: The speaker considers the dynamic workflows release somewhat unsuccessful and believes, based on their understanding, that it is not used often.

“i've tried it and like it doesn't act in the way that like again i i'm not i'm not the biggest open ai like”

Claim: The speaker says that after trying the system, its behavior did not match what they expected.

“you know stand or something but i think whatever they did was very impressive like they somehow managed to get away for this this swarm to actually act towards towards a goal and yeah it's very difficult yeah”

Claim: The speaker considers OpenAI’s work impressive because it enabled an agent swarm to act toward a goal.

“for granted what it means for a swarm to converge to an answer”

Claim: The speaker believes people underestimate the difficulty of getting an agent swarm to converge on an answer.

“i i think it's a little bit blown out of proportion like gemini again so i should preface by saying i haven't worked at any of these places so so take this with it with a grain of salt”

Claim: The speaker thinks Gemini’s work has been somewhat overhyped, while qualifying the opinion because they have not worked at any of the companies under discussion.

“um let me just say this first this work was really impressive i think what what they showed here was like they took a a time when the models weren't that good and they managed to be very very smart about like what the harness does i remember for this at least like yeah like alpha geometry i guess that was a year before this but it was very cool they took um they took it to the max and they like designed uh i don't know i'd say i i'm not too big on competitive math but i think like gdm it's sort of a shame like everyone i've talked to about gdm kind of has the same opinion which is that”

Claim: The speaker considers the work impressive because the team designed the harness intelligently despite the models being relatively weak at the time.

“but it's way too like bureaucratic whatever is good they have the talent and the resources to do almost anything but like i don't know until they figure that part out like nothing against anti-gravity for example but like i don't know anybody that uses anti-gravity and so i've tried it once and it's i don't see a reason to switch to it um and i think for whatever reason like they've been struggling with this so yeah”

Claim: The speaker believes the organization has ample talent and resources but is hindered by excessive bureaucracy; they have not encountered users of Antigravity and, after trying it once, see no reason to switch to it.

“okay let's let's talk about mismanaged news for a little bit i the only comment i'll say on speculative ptc is that it's a really simple idea it's almost like obvious that this should be done and like there's not much more to talk about it like i think it's just like you should just use it for for like coding like anything with programmatic agent calling like rlms or code act like yeah it's like it's like a no-brainer”

Claim: The speaker considers speculative PTC a straightforward, nearly obvious technique that should be applied to coding tasks involving programmatic agent calls, including RLMS or CodeAct.

“thing is when the model is uh writing its code or like even as like after it finishes writing code a lot of tools tend to be like sequential or like you have to wait on them so you should just launch them in advance like if if you're able to compile this code you can probably figure out like even though it's like kind of variables and stuff like you can figure out like”

Claim: The speaker argues that coding agents should launch tools in advance when the code makes the required execution predictable, because tools are often sequential or involve waiting.

“yeah so yeah there's there is this uh someone pointed to me some actually like academics have especially pl like programming languages people have some like very kind of cool ways of doing this and so like you know at some point maybe i'll i'll like work on this guess mostly”

Claim: The speaker says academics, particularly programming-languages researchers, have developed promising approaches to advance tool execution, and the speaker may work on the problem in the future.

“should do this like really just sit down and think about like if i took astra even the current frontier models are not good enough at like doing a particular job over let's say the span of a month consistently and well and i think this is like a stupid problem like i i genuinely think we can solve this you don't need to be a frontier lab and like do all this like you know fancy stuff for your ipo like i think like”

Claim: The speaker believes the problem of getting models to perform a job consistently for a month can be solved without being a frontier lab or using elaborate techniques.

“should do this like really just sit down and think about like if i took astra even the current frontier models are not good enough at like doing a particular job over let's say the span of a month consistently and well and i think this is like a stupid problem like i i genuinely think we can solve this you don't need to be a frontier lab and like do all this like you know fancy stuff for your ipo like i think like”

Claim: Current frontier models cannot yet perform a particular job consistently and well for a month.

“like these models are so smart that even if it's like a silly way i think that it genuinely is a skill issue of you can get a model to be as good as let's say like just some 18 year old high school kid”

Claim: The speaker believes models can be made as capable as an 18-year-old high-school student at some job, even with an unsophisticated approach.

“doing some job i think it's like ridiculous that we can't do that and it's part of the reason is like the the format of a language model is not really amenable to that but i think you can shape a harness around it and do it and i think like this in itself is like ignoring the rlm stuff ignoring all the like you know what abstraction should we use like i just think someone can design a harness that can do this like i i i mean that's maybe”

Claim: The speaker believes language models’ format is poorly suited to reliably performing jobs, but a purpose-built harness can enable them to do so.

“doing some job i think it's like ridiculous that we can't do that and it's part of the reason is like the the format of a language model is not really amenable to that but i think you can shape a harness around it and do it and i think like this in itself is like ignoring the rlm stuff ignoring all the like you know what abstraction should we use like i just think someone can design a harness that can do this like i i i mean that's maybe Do long running but simple tasks and do them reliably.”

Claim: The speaker believes a harness can be designed to make long-running, simple tasks reliable.

“So, so for example, like if I wanted. a general system that i could kind of talk i can talk to it like i would talk to an intern and basically just ask it to do to explore some small things so maybe an example of this is like very silly auto research is maybe an example of this of like not necessarily finding super super novel solutions but at least optimizing all of the easy parts of”

Claim: The speaker envisions a general-purpose system that can be directed conversationally to explore small problems, such as automating research to optimize easy parts of a problem without necessarily finding highly novel solutions.

“people's general workflows where you probably could just vibe code up some specific harness to help you do like automate this thing some examples are like automating finding like research papers and stuff like that but usually people will design like a specialized agent to help them do this kind of thing or like like they'll vibe code a harness like and just run it or like their slack slack bot or something but i almost think there's just like a standard form like just a harness that you just plug in like you don't it doesn't need to be designed for finding papers or like you just kind of tell it find this for me and you like plug it into that setting what i'm getting at is that i think there's a lot of easy things that can be automated and”

Claim: The speaker believes many easy tasks in people’s workflows could be automated with a reusable general-purpose harness that users can direct without designing a specialized agent for each task.

“is the easiest form of this but what this is kind of saying is that like we have jagged intelligence on a lot of things like for example models are like disproportionately good at coding and math this is saying like we can translate those abilities to many other things so like”

Claim: The speaker says models have disproportionately strong coding and math abilities and argues that these abilities can be applied to many other domains.

“I don't actually know this applies to models. For example, like in GPU code optimization, one very interesting question is whether if you were to take out all of the GPU programming data like from a model. but it was like as good as Astra is now just without like without taken out would it be able to still optimize GPU kernels like would it be able to learn in context roughly what it needs to learn and then like have some pipeline or like or come up with some solution to solving like optimization tasks and I think like there's like a mismatch between like if you took a human that was as smart or like knew as much Astra there is a mismatch between what that human can do and what Astra”

Claim: The speaker is uncertain whether human-like transfer across problem-solving domains applies to models and asks whether Astra, without GPU-programming training data, could learn in context to optimize GPU kernels.

“And I think we can actually approximate the human.”

Claim: The speaker believes models can be made to approximate human performance substantially more closely.

“a category theory type of abstraction where you're thinking categories and then you have to like then translate down to the specific and but then you like actually really care more about the category and like that that's the communication error because like everyone's listening for the specific but actually trying to also you know convey the general which is hard”

Claim: Communicating a general abstraction is difficult when listeners focus on specific details.

“because it's hard like you're compressing a lot into word after like sequential word decoding.”

Claim: Sequential word-by-word decoding makes it difficult to compress a large amount of information into speech.

“how long english has been around which is let's say 500 years uh which is not that long like actually uh you know like what you what the language you speak constrains how you think and if you learn a different language uh for example uh someone in chinese we don't have tenses um i don't know if you i i actually didn't know that and i speak chinese”

Claim: The language a person adopts affects how that person thinks.

“whatever like anyway so like the the language that you adopt affects your thinking and if you choose to output your chain of thought in english you are biasing towards whatever english solves i don't know what that sort of prior of english is”

Claim: A model that outputs its chain of thought in English is biased toward the kinds of problems or ideas English expresses well.

“also like let's say german like you know subject object agreements you have to put the verb at the end which is very super annoying like very famously”

Claim: In German, the verb comes at the end of the sentence, so the listener may not know the sentence's meaning until reaching the verb.

“uh yeah right you don't know what you're doing until the end where you're like oh that mess of nouns and then the verb well the most classic one that most people be have heard of is arrival where um they have the heptapods where they they think uh their time is like flat to them so they think in they they they output entire sentences at one shot um so it's as closest to the difference between autoregression and diffusion we talk in autoregression”

Claim: In the film Arrival, the heptapods are depicted as experiencing time as flat and producing whole sentences in one shot.

“and what if you could talk in diffusion where things just resolve over time but like the whole idea shows up at once so that is a drastically different language but it”

Claim: The speaker proposes that a diffusion-like mode of communication could make an entire idea appear at once as its components resolve over time.

“that's really interesting which like machines could speak that we you know probably would never speak but like yeah machines don't care yeah”

Claim: Machines could use languages that humans probably would never speak, because machines do not care whether a language is human-like.

“uh programming language theory uh where everything is like pure functional and like completely relational and and you sort of abstract away the solver that translates the relationships that is always true into code so yeah I feel like this is maybe a little bit too out”

Claim: Some programming-language-theory approaches use pure functional, relational formulations and abstract away a solver that translates relationships that are always true into code.

“but I love languages whether it's coding or human and I do think a lot about how that affects reasoning and the boundaries of what we can do I don't need to go too much beyond that I don't know if you have any other thoughts my closing question was going to”

Claim: The speaker is interested in both programming and human languages and thinks about how language shapes reasoning and the limits of what people can do.

“yeah i guess in in the sense that like so you did bring up that like i i like to think about things in like a math oriented way it's like very uncomfortable sometimes to be working on like harnesses and agents because it's so”

Claim: Working on harnesses and agents can feel uncomfortable to the speaker because they prefer thinking about problems in a math-oriented way.

“it yeah it's it's it's also just like like empirically it's hard to like verify a lot of findings at least with with the compute that we have available to us but the reason why i think i've moved on to a lot of these problems is i think actually this is where most of like the innovation is yet to happen to me like the gpu level is a means to”

Claim: Alex Zhang believes that most innovation in the problems he has moved on to is still ahead.

“it yeah it's it's it's also just like like empirically it's hard to like verify a lot of findings at least with with the compute that we have available to us but the reason why i think i've moved on to a lot of these problems is i think actually this is where most of like the innovation is yet to happen to me like the gpu level is a means to”

Claim: Alex Zhang views GPU-level work as a means to a broader research goal, rather than an end in itself.

“it yeah it's it's it's also just like like empirically it's hard to like verify a lot of findings at least with with the compute that we have available to us but the reason why i think i've moved on to a lot of these problems is i think actually this is where most of like the innovation is yet to happen to me like the gpu level is a means to”

Claim: Alex Zhang says that many findings are difficult to verify with the compute currently available.

“you exploring other ideas like you you you want to for example like get good at writing kernels or like even automate writing kernels for the sake of a broader goal of like i want to explore ideas where i'm not bottlenecked by systems challenges in that sense like you know i guess a lot of what's written there is all harness stuff but i am also interested in things at the model level as well um but i'll just leave it at that yeah”

Claim: Alex Zhang wants to explore research ideas without systems challenges becoming a bottleneck, and is interested in both harness-level and model-level work.

“I guess there's nothing I have in particular where I feel like I need to work with someone on unless it's like unless it's with a company before like compute or like with to talk with other people about it but I will say I'm not I'm never opposed to working on ideas with other people I get reached out to a lot by often undergrads or even like other students podcasters podcasters um and usually I feel”

Claim: Alex Zhang is open to collaborating with other people on ideas.

“I guess there's nothing I have in particular where I feel like I need to work with someone on unless it's like unless it's with a company before like compute or like with to talk with other people about it but I will say I'm not I'm never opposed to working on ideas with other people I get reached out to a lot by often undergrads or even like other students podcasters podcasters um and usually I feel”

Claim: Alex Zhang does not have a specific collaboration he feels he needs, except possibly collaborating with a company on compute or discussing compute with others.

“they'll like they'll be like i read your paper in quotes like recursive language models or like prime agent like a self-improving rlm harness or something uh and it's kind of like i i i like i really like people that are opinionated even if we disagree i think if you have strong opinions and are able to like think through why you think those opinions are right or wrong because usually it's it's hard to to actually tell but like you have strong convictions about certain problems like i'm always happy to like chat and even like potentially work on something together i have like no limit to who or like what i would like to work on so um yeah”

Claim: Alex Zhang welcomes conversations and potential collaborations with people who hold strong opinions and can explain their reasoning, even when they disagree.

“yeah yeah yeah i i you know in in the era of agents i think there's a lot more work you can do you know like like bandwidth wise so i yeah i i think in general like i am not hard to impress but i think it just takes a little bit of effort to kind of”

Claim: Alex Zhang thinks that, in the era of agents, people can take on more work because of increased bandwidth.

“yeah yeah yeah i i you know in in the era of agents i think there's a lot more work you can do you know like like bandwidth wise so i yeah i i think in general like i am not hard to impress but i think it just takes a little bit of effort to kind of”

Claim: Alex Zhang says he is relatively easy to impress, though doing so takes some effort.

“guess maybe that one of the smaller differences in like i mean actually maybe you're doing research i'm not sure but for for me at least like i'll have maybe like 10 or 15 different ideas that i want to do but the thing is like most of them are bad and and this also maybe is true even for someone that reaches out to me like maybe the idea is actually”

Claim: Alex Zhang may have 10 to 15 ideas he wants to pursue, but he considers most of them bad.

“guess maybe that one of the smaller differences in like i mean actually maybe you're doing research i'm not sure but for for me at least like i'll have maybe like 10 or 15 different ideas that i want to do but the thing is like most of them are bad and and this also maybe is true even for someone that reaches out to me like maybe the idea is actually is actually bad but it looks interesting to me and so like we can spend like”

Claim: Alex Zhang recognizes that an idea may seem interesting to him while actually being bad.

“some time looking into it and if like we feel like there's actually something there like then we should we should take the next few weeks and just really pursue it and like this is my my style with this is why i love the phd by the way because there are times when i'm just thinking about problems like maybe on a run or just like playing tennis or something like i'm not i'm not working i guess but it's like those are the most fun times and then when i like really am convicted about something i'll just like drop everything and just do it”

Claim: If Alex Zhang and a collaborator find an idea promising after investigating it, he thinks they should pursue it intensively for the next few weeks.

“some time looking into it and if like we feel like there's actually something there like then we should we should take the next few weeks and just really pursue it and like this is my my style with this is why i love the phd by the way because there are times when i'm just thinking about problems like maybe on a run or just like playing tennis or something like i'm not i'm not working i guess but it's like those are the most fun times and then when i like really am convicted about something i'll just like drop everything and just do it”

Claim: Alex Zhang enjoys his PhD partly because thinking about problems while running or playing tennis is among the most enjoyable parts of his work.

“some time looking into it and if like we feel like there's actually something there like then we should we should take the next few weeks and just really pursue it and like this is my my style with this is why i love the phd by the way because there are times when i'm just thinking about problems like maybe on a run or just like playing tennis or something like i'm not i'm not working i guess but it's like those are the most fun times and then when i like really am convicted about something i'll just like drop everything and just do it just spend like all my time thinking and working on that problem and then i you know once you get to the point where like you can just run experiments then it's it's it's kind of easy coasting again so yeah”

Claim: When Alex Zhang becomes strongly convinced that a problem is worth pursuing, he sets aside other work and focuses on it.

“just spend like all my time thinking and working on that problem and then i you know once you get to the point where like you can just run experiments then it's it's it's kind of easy coasting again so yeah”

Claim: Alex Zhang says that once a research problem reaches the point where experiments can be run, the work becomes relatively easy.

“I actually worked on like AI for bio stuff before I started my PhD.”

Claim: Alex Zhang worked on AI for biology before beginning his PhD.

“The field has changed a lot since then, I should say, because it's like it used to”

Claim: Alex Zhang says AI for biology has changed substantially since his pre-PhD work in the field.

“Yeah, you know, I will preface by saying my understanding of a lot of these topics is probably pretty limited.”

Claim: Alex Zhang believes his understanding of many science topics is probably quite limited.

“But I think like, if I find out either because someone reaches out or like I look at a problem and I'm like, like some design principles that we use or that we're thinking about right now actually make a lot of sense for this problem. i get excited about those as well but i think i think it's harder i i don't know i think with i think science especially like empirical or like applied science has very very long like what is it called like feedback loops or whatever um yeah yeah”

Claim: Alex Zhang gets excited when design principles his group uses or considers appear applicable to a problem in another field.

“i get excited about those as well but i think i think it's harder i i don't know i think with i think science especially like empirical or like applied science has very very long like what is it called like feedback loops or whatever um yeah yeah”

Claim: Alex Zhang thinks empirical and applied science have very long feedback loops.

“yeah i mean to me like also this aspect of like what is worth spending and betting my time on now because like maybe i spent a lot of time on this problem and then like in six months like a different solution kind of like maybe a new model comes out and it's like oh it's way better for this and so i do have to be careful i like you know you have to be conscious about like where you think things might be going um so yeah”

Claim: Alex Zhang says he must be careful about which problems he invests time in because a different solution or a new model might emerge within six months and work much better for a problem.

“yeah i mean arcade gi3 got saturated in less in less than a year or so it's kind of you know like it's i i don't know like if if if you were a lab picking that problem like you're probably kind of sad now because astro”

Claim: Alex Zhang says arcade games became saturated in less than about a year.

“in less than a year or so it's kind of you know like it's i i don't know like if if if you were a lab picking that problem like you're probably kind of sad now because astro”

Claim: Alex Zhang says a lab that chose the arcade-game problem would probably be disappointed now, given Astro.

“no i i will say because i think i think especially at mit like it's it's there's a lot of really talented scientists there like in in the natural sciences and i think it's it's a little bit like sacrilegious almost to be like i'm gonna figure out like your problem like this”

Claim: Alex Zhang says it feels somewhat sacrilegious to claim that he will solve a natural scientist’s problem at MIT.

“no i i will say because i think i think especially at mit like it's it's there's a lot of really talented scientists there like in in the natural sciences and i think it's it's a little bit like sacrilegious almost to be like i'm gonna figure out like your problem like this”

Claim: Alex Zhang says MIT has many highly talented scientists in the natural sciences.

“well yeah i mean i i think that that uh that whole thing is kind of weird because i feel like i feel like they would have had a stronger case if a lot of them them didn't work with open ai before like all this happened”

Claim: Alex Zhang thinks the mathematicians opposing AI use would have had a stronger case if many of them had not previously worked with OpenAI.

“paper is more about like empirical results. Like the actual idea is quite simple.”

Claim: Alex Zhang says the RLM paper focuses more on empirical results and that its underlying idea is quite simple.

“Yeah, yeah, yeah. At this point, I think there's more interesting things to look over now.”

Claim: Alex Zhang thinks there are more interesting things to examine now than the RLM paper.