[One Million AI Engineer Special] How Stanford Teaches AI-Native Engineering — Mihail Eric, Monaco

“Yes. This is the second time it's being taught, and I taught it for the first time last fall, so fall 2025. This was pre like Opus four five, four six days.”

Claim: Mihail Eric taught the course for the first time in fall 2025, and the current offering was its second time being taught.

“But the-- even before fall 2025, the course had been about, like, a year in the making. The genesis of the class was like November of 2024, and I had sort of had this idea that we're seeing all this stuff happening in coding agents, and yes, they were an earlier version of the coding agents, but already it felt like there was something drastically changing in terms of how developers were doing their work, how quickly we were getting work done, uh, all that fun stuff.”

Claim: Around November 2024, Mihail Eric conceived the course after observing that coding agents were changing developers’ work practices and the speed at which they completed work.

“And so at the time, I just-- it was all just scattered blog posts, scattered, like, things you could find online if you were trying to really get into the, the methodology of how to do these things.”

Claim: Before Mihail Eric developed the course, guidance on coding-agent methodology was scattered across blog posts and other online sources.

“And so I, I had this idea like, "Hey, well, is there a way to kind of structure a curriculum around this that's a little bit more formal?" Had the idea. And you know, I, I-- my alma mater is Stanford, and I always had this dream to go back and teach something and, and kind of pay it forward.”

Claim: Mihail Eric wanted to create a more formal curriculum for coding-agent practices.

“And so I, I had this idea like, "Hey, well, is there a way to kind of structure a curriculum around this that's a little bit more formal?" Had the idea. And you know, I, I-- my alma mater is Stanford, and I always had this dream to go back and teach something and, and kind of pay it forward. Uh, I've always been very deep into education. I talked to Chip, who also taught a class, uh, one of my good friends.”

Claim: Mihail Eric had long been deeply interested in education and wanted to return to Stanford to teach and give back.

“Yeah, Chip Hieatt, exactly. Uh, spoke to her. Who-- she had taught her, I guess, designing machine learning systems. And so, like, asked her what that process was like, and sort of, you know, took a lot of inspiration for how she-- from how she designed the curriculum and, and thought about, you know, getting something through the, through the school.”

Claim: Mihail Eric consulted Chip Hieatt about her teaching experience and took inspiration from her curriculum-design process and approach to getting a course approved at Stanford.

“Put it together, spent about nine to ten months. This was now fall of twenty twenty-four through to like, you know, mid to late of twenty twenty-five. Got it through the committee, uh, made a case for the class to exist.”

Claim: Mihail Eric spent about nine to ten months developing the course and obtaining committee approval.

“You know, I had to put together a proposal and everything. Uh, and that took me to fall twenty twenty-five when I taught it for the first time, which was very well-received. You know, students were really interested in it.”

Claim: When Mihail Eric first taught the course in fall 2025, it was very well received and students showed strong interest.

“I would say really coding agents, autonomous coding agents really have picked up- ... in a way that, that we had not seen before. Um, so it felt like it was time to do this again, uh, refresh things, uh, and then here we are.”

Claim: Mihail Eric said autonomous coding agents had gained momentum in an unprecedented way, prompting him to refresh and teach the course again.

“There was, like, a coding agent for code review. There was a coding agent for, you know, spec design. There was a coding agent for, uh, SRE. And, you know, some of those-- of course, some of those companies still exist, but it felt like last year it was a lot more about a broad tour of here's everywhere you can use coding agents. And in a way, a lot of the curriculum last year was, was geared toward exposure and this, this kind of survey, this kind of like odyssey of sorts through, hey, everything that you knew a-a-about the ISDLC is now changing because of AI.”

Claim: The speaker says last year's class broadly surveyed applications of coding agents across the software development lifecycle.

“Um, this year, I think there's some principles that are solidifying, right? The idea of, like, very careful context engineering. The idea of, like, really thinking through sort of like design and architecture as a way of handling really complex tasks.”

Claim: The speaker says principles for AI-assisted software engineering are solidifying, particularly careful context engineering and using design and architecture to manage complex tasks.

“And so the nature of my class is it's not intended for the, like, beginner programmer. It is not a, it is not a replacement for, like, traditional computer science in the sense of, like, algorithms and if-- in fact, a lot of those classes are prereqs to what I'm, what I'm doing here.”

Claim: The speaker's class is intended for students with prior computer science preparation, not beginner programmers, and does not replace traditional computer science; many such courses are prerequisites.

“You know, at Monaco, we deal with ma-massive code bases. You know, I help a lot of companies also with thinking through their massive code base, uh, designs and things like that. And so really, it's more of a practitioner's course for once the students have graduated or are now entering the workforce, how to be the most productive members of, of the workforce.”

Claim: The speaker describes his course as a practitioner-oriented course for graduates and workforce entrants, aimed at helping them become highly productive employees.

“Yeah. I, uh, you know, there's a lot of anxiety from students, as you can probably imagine. And, uh, and I, I think a lot of what I'm trying to communicate this time around is that the anxiety, a lot of it still is here, and there was anxiety last year as well.”

Claim: The speaker says students are experiencing substantial anxiety this year, as they also did last year.

“But I wanna make the case that A well-trained AI native engineer, and I know that's like a buzzy word- ... but let's say that we'll, we'll, we'll, we'll, we'll dive into maybe what aspects of that, that, that are most interesting. A like well-trained and native engineer can be some of the most, you know, the most useful assets like any company around. And so, and, and, you know, this is the first week of class actually, and like week one, and you know, first lecture I'm like, "Hey, I really believe that every company needs you," but the floor is much higher, right?”

Claim: The speaker believes a well-trained AI-native engineer can be one of the most useful assets a company has.

“A like well-trained and native engineer can be some of the most, you know, the most useful assets like any company around. And so, and, and, you know, this is the first week of class actually, and like week one, and you know, first lecture I'm like, "Hey, I really believe that every company needs you," but the floor is much higher, right?”

Claim: The speaker says the expected baseline for new software engineers entering the workforce has risen substantially.

“So you're entering as like your first job out of school, but it's not sufficient to just be like a, you know, an L3, in like a Google term, if you're gonna use like that, that kind of a leveling system there.”

Claim: The speaker says that an engineer entering a first job after graduation must be capable of more than the L3 level of performance.

“It's like you have to generally be better than, you know, almost at the senior level going into this. So like what, what can we-- how do we get there more quickly, right? And so, uh, I'm trying to say that there is still a core bit of like foundational software engineering that is important to have some of these core values of, of sort of how software systems are built and the trade-offs of those systems.”

Claim: The speaker says new engineers generally need to enter the workforce nearly at senior level, while retaining foundational software engineering knowledge about system construction and trade-offs.

“But then beyond that, we go and, uh, you know, that's where I think you as the humans should still be valuable rather than... And the rest, the implementation details, you can kind of defer more often.”

Claim: The speaker believes human engineers should focus on areas beyond foundational software engineering, while deferring implementation details more often.

“And I'm saying I'm a practitioner, a-as, as you pointed out, Swyx. Like I, I'm a practitioner. I do this thing on my, on, you know, my day-to-day. I design these systems. You know, we build, we build these things at Monaco and have seen that many times before.”

Claim: Mihail Eric is a practitioner who designs systems and builds systems at Monaco.

“And so that, that being said, I don't wanna claim that I have all the answers today, and that would be, that would be like very, uh, you know, be a lot of hubris to, to make that claim.”

Claim: Mihail Eric does not claim to have all the answers, saying that doing so would be hubris.

“And so that, that being said, I don't wanna claim that I have all the answers today, and that would be, that would be like very, uh, you know, be a lot of hubris to, to make that claim.”

Claim: The speaker says he does not claim to have all the answers about AI-native engineering and considers such a claim hubristic.

“So the structure of the class was always, here's one practitioner's sort of aggregated set of insights from working in the field, from like talking to a lot of people in the field.”

Claim: Mihail Eric structured the class around one practitioner's aggregated insights from fieldwork and conversations with people in the field.

“So the structure of the class was always, here's one practitioner's sort of aggregated set of insights from working in the field, from like talking to a lot of people in the field.”

Claim: The speaker says his class presents an individual practitioner's synthesis of insights from practical work and discussions with others in the field.

“But then there's people that are like literally on the frontier, you know, seeing things even beyond that, and invite them to sort of put their p- their, their proposals forward, their view of the wor- w- the world forward, and that's why the guest lectures are there.”

Claim: Mihail Eric included guest lectures so people working at the frontier could present their proposals and perspectives.

“But then there's people that are like literally on the frontier, you know, seeing things even beyond that, and invite them to sort of put their p- their, their proposals forward, their view of the wor- w- the world forward, and that's why the guest lectures are there.”

Claim: The speaker says guest lectures let people working at the frontier share their proposals and perspectives.

“That's right. That's right. Yeah. Yeah. Exactly. And, and I, but I do think that there's a balance for, for this kind of material to, to your point is that there is a-- You can't-- It's not 100% solidified.”

Claim: Mihail Eric says the curriculum for this material is not completely settled.

“I mean, by, by design. I mean, by design, it's a feature and maybe a bug of some, some kind, but a, a big portion of the curriculum did change from last year, right? And so the, the, there's like a dynamism to the whole development of the curriculum, but I'm trying to at least put forward that there are some things that I believe are kind of sticking around, and yes, prompting coding agents, they've gotten smarter.”

Claim: Mihail Eric believes prompting coding agents is among the course elements likely to persist, despite coding agents becoming smarter.

“I mean, by, by design. I mean, by design, it's a feature and maybe a bug of some, some kind, but a, a big portion of the curriculum did change from last year, right? And so the, the, there's like a dynamism to the whole development of the curriculum, but I'm trying to at least put forward that there are some things that I believe are kind of sticking around, and yes, prompting coding agents, they've gotten smarter.”

Claim: A large portion of Mihail Eric's course curriculum changed from the previous year by design.

“What does that mean in terms of the new ways of dealing with them? But then still, Boris is also gonna know something that I don't know, right? So let's, let's have, let's have him also come to the table and, and share his learnings.”

Claim: Mihail Eric says Boris has knowledge he lacks and should be invited to share his learnings.

“Basically, the idea was... So last year when I taught the class, we had like nine assignments, and then we had like a capstone open-ended project, right? The nine assignments were basically me sort of like coming up with these pedagogical exercises, like, "Oh, here's a repo.”

Claim: Mihail Eric's class had nine assignments and an open-ended capstone project the previous year.

“Basically, the idea was... So last year when I taught the class, we had like nine assignments, and then we had like a capstone open-ended project, right? The nine assignments were basically me sort of like coming up with these pedagogical exercises, like, "Oh, here's a repo. Like use an AI code review tool to, to, you know, post a code review and then see what-- why is it good, why is it not good." So relatively pedagogical. Honestly, pretty easy.”

Claim: The previous year's assignments in Mihail Eric's class were pedagogical exercises, including using an AI code-review tool to review code in a repository.

“Like use an AI code review tool to, to, you know, post a code review and then see what-- why is it good, why is it not good." So relatively pedagogical. Honestly, pretty easy.”

Claim: Mihail Eric considered the previous year's assignments relatively pedagogical and pretty easy.

“I think for a lot of folks, they could've just like one-shot it, and I thought, "Okay, this is not that helpful." At this point, agents are around. I mean, they, they have every student that I surveyed, you know, on Tuesday, not a single person really types by, by hand anymore. They're like- ... everyone's using coding agents. There's no, there's-- And this is, you know, like all sophomores, juniors at, at Stanford.”

Claim: In a survey of his Stanford sophomore and junior students, Mihail Eric found that everyone was using coding agents and no one really typed code by hand anymore.

“I think for a lot of folks, they could've just like one-shot it, and I thought, "Okay, this is not that helpful." At this point, agents are around. I mean, they, they have every student that I surveyed, you know, on Tuesday, not a single person really types by, by hand anymore.”

Claim: Mihail Eric thought the previous assignments were not very helpful because many students could have completed them in one shot.

“Okay. I haven't yet, but that's a good call.”

Claim: Mihail Eric had not yet surveyed his students about which coding-agent tools they use.

“Right. Right. So I think by default, even before the class, there's like, you know, a Claude license. I, I imagine Cursor license. There's like probably a Cognition license as well.”

Claim: Mihail Eric thinks students may have access through the university to Claude and likely Cursor licenses, and possibly a Cognition license, even before the class.

“I don't know, you might know better than I do. But what I was able to also just accrue on top of that was, hey, like if I don't want people to be bottlenecked by like the $20 a month subscription, like I want you guys to go, if you wanna like token max something just to see what's possible, then, then you should. And so, and I'll come back to the, the partners, but the, the additional thing I was able to, to accrue through sort of talking to folks was like additional credits for students.”

Claim: Mihail Eric obtained additional credits for students through conversations with people.

“There, there, there's almost like this sense of like them being left behind, and I think that mentorship is like a really important kind of piece of just up-leveling everyone and making sure that there isn't this like massive gap that folds, that, that occurs between, you know, the senior people that have already been in the industry before coding agents and know what's up, and they adopt coding agents, and they become amazing, and then the juniors that are just like, "Okay, now what do we do?"”

Claim: Mentorship is important for helping students advance and preventing a large gap between experienced developers who adopt coding agents and junior developers who may be left behind.

“So in a sense, one of the, the kind of goals was, hey, coding agents are really good. I wanna put you guys in situations where you're dealing with complex code bases, large code bases at scale.”

Claim: The course aims to have students work with complex, large codebases at scale, in light of coding agents' capabilities.

“We're gonna work with some of like what I think are the-- among the best like AI native repos, you know, that all do AI-focused things or either coding agents themselves or building tools in the service of agents, you know, in the modern world.”

Claim: The course will work with repositories the speaker considers among the best AI-native repositories, including projects focused on AI, coding agents, or tools that serve agents.

“I want you guys to get involved with these projects. I want part of your assignments to be actually contributing to these projects. In the course of being in all those projects, you'll actually be talking to the maintainers.”

Claim: The course assignments are intended to have students contribute to AI-native projects and interact with their maintainers.

“They will be reviewing your code. They'll be helping you build mental models of, like, good software. In that way, there's effectively this, like, passing on of, like, the guard of true good software design in a way that, you know, I don't think most students get the opportunity to do.”

Claim: In the course projects, maintainers will review students' code and help them develop mental models of good software, providing an opportunity to learn good software design that the speaker believes most students lack.

“And then there's the benefit, of course, that, like, I'm sure some of them will get, you know, poached and picked up as part of, uh, doing well in those assignments.”

Claim: The speaker expects that some students will be recruited as a result of performing well on the course assignments.

“Yeah, yeah. And I think that there's-- I still have some assignments thrown in there, which I think are not intended to be the hardest things in the world, but are meant to be more about like concept, concepts, like more conceptual things.”

Claim: Some course assignments are designed to emphasize conceptual understanding rather than being among the hardest tasks.

“Like, you know, what does a skill mean? It's sort of like the, maybe more of the, the, the tactical of some of these tools, skills like and, you know, agent-ready code bases, like agents, MDs, things like this.”

Claim: The course includes introductory material on concepts and practices such as skills, agent-ready codebases, and AGENTS.md files so students unfamiliar with them can get up to speed.

“So there's a, a certain amount of just, "Hey, if you haven't heard these terms, kind of get up to speed on them." I do think most of the learning, though, in terms of the kind of putting the tools to the test will happen both in, in the OSS and then also in the, like, the final project, where it's, like, very open-ended, and they'll have to just kind of explore something interesting.”

Claim: Most hands-on learning with the tools in the course is expected to occur through open-source projects and an open-ended final project in which students explore a topic of interest.

“Yeah, yeah. So, uh, week one, it's all, uh, internals of harnesses, you know, kind of build a coding agent from scratch. Uh, week two is more, okay, now assume that there's, uh, how do we start thinking about, like, context engineering at scale? Uh, so beyond just vanilla prompting. Uh, and then I think coupled with that is just, um, in, in doing that, thinking about, like, tool design, MCP, you know, people-- I think for a while MCP was getting a lot of hate.”

Claim: The course’s second week covers context engineering at scale beyond vanilla prompting.

“Yeah, yeah. So, uh, week one, it's all, uh, internals of harnesses, you know, kind of build a coding agent from scratch. Uh, week two is more, okay, now assume that there's, uh, how do we start thinking about, like, context engineering at scale?”

Claim: The course’s first week teaches students about agent-harness internals and how to build a coding agent from scratch.

“Uh, so beyond just vanilla prompting. Uh, and then I think coupled with that is just, um, in, in doing that, thinking about, like, tool design, MCP, you know, people-- I think for a while MCP was getting a lot of hate. I'm still pro MCP. I see it all the time, and so I think that there's, at this point, it's a standard that's existed, and I think it will continue to exist. So, you know, talking about that, effective tool design, uh, that's week two.”

Claim: The course’s second week covers effective tool design and MCP.

“I'm still pro MCP. I see it all the time, and so I think that there's, at this point, it's a standard that's existed, and I think it will continue to exist. So, you know, talking about that, effective tool design, uh, that's week two.”

Claim: The speaker supports MCP and expects it to continue to exist as a standard.

“Uh, week three I would say is more about just like, uh, like skills and, like, what does it mean to, like, extend beyond simple single-purpose tools to more, like, procedures, and that's where we'll talk about skills and, you know, that kind of starts leading into, like, the automation of agents.”

Claim: The course’s third week covers agent skills and procedures that extend beyond single-purpose tools, leading into agent automation.

“Um, week four is, is a lot of just more of, like, the foundational pieces of how can you customize an agent, and this is things like agent config files, a la agents MB, Cloud MD, hooks, um, you know, kind of sub-agents.”

Claim: The course’s fourth week teaches agent customization, including configuration files, hooks, and sub-agents.

“Like, there's a lot of mechanisms that exist in modern harnesses, uh, which you can use for different purposes, so I want to expose students to that. Agent-ready code bases, this is, uh, probably one of my favorite topics.”

Claim: Modern agent harnesses provide many mechanisms for different purposes, and the speaker wants to expose students to those mechanisms.

“It's like-- A-and this is especially relevant for folks that are going into, like, industry and thinking about code at scale is, you know, essentially, what does it mean to have a repo that's ready to be ingested, consumed effectively by an agent?”

Claim: The course’s fifth week examines how to design code repositories so agents can ingest and use them effectively.

“And so thinking a lot about, you know, the trade-offs of designing of kind of service level boundaries, but also docs and, and verifiability and things of that nature. Um, so that's week five.”

Claim: The course’s fifth week considers trade-offs involving service-level boundaries, documentation, and verifiability in agent-ready codebases.

“Uh, yeah, week six, uh, sort of agentic code review, um, and here I think what is interesting is thinking also kind of extending from week five, week five around code base, like agent-ready code bases.”

Claim: The course’s sixth week covers agentic code review.

“Going to week six is more about, like, okay, in order to... Like, code review is now becoming a bottleneck, which I don't know that students, unless they've, you know, have done a lot of-- Some of them are master's students, so they've been in the industry, but a lot of them have not seen this. But for most people that are using code agents today, we're all experiencing something of a new bottleneck in code reviews, so how do we unblock that or sort of un-bottle the, the bottleneck a little bit?”

Claim: The speaker says code review is becoming a bottleneck for people using code agents.

“Um, what mechanisms are there there? And so verifiability and, you know, things of that nature and how we can extend that into the code review process. Week seven, uh, security. I mean, obviously, I think there was a point-- So this was in last year's class as well, and this, this I wanted to reinforce. I mean, with any autonomous system taking a lot of actions, we're seeing a lot of, this year especially, a lot of things getting hacked. Um, you know, not least of which are, are just some of the, the core pieces of software from, from Vercel to, you know, uh, gateways, LM gateways entirely. So I want to spend some time just, like, talking about what new attack vectors exist, exist in security and, and, you know, how we can mitigate some of them.”

Claim: The course’s seventh week covers security, including attack vectors affecting autonomous systems that take many actions.

“Um, what mechanisms are there there? And so verifiability and, you know, things of that nature and how we can extend that into the code review process. Week seven, uh, security.”

Claim: The course’s sixth week considers mechanisms for extending verifiability into code review.

“I mean, obviously, I think there was a point-- So this was in last year's class as well, and this, this I wanted to reinforce. I mean, with any autonomous system taking a lot of actions, we're seeing a lot of, this year especially, a lot of things getting hacked. Um, you know, not least of which are, are just some of the, the core pieces of software from, from Vercel to, you know, uh, gateways, LM gateways entirely.”

Claim: The speaker says that many systems have been hacked this year, including core software and LLM gateways.

“So I want to spend some time just, like, talking about what new attack vectors exist, exist in security and, and, you know, how we can mitigate some of them.”

Claim: The speaker wants the course to address ways to mitigate new security attack vectors.

“Uh, week eight is background agents, so this is, uh, what I want to start touching on here is, you know, how do you, like- Basically run agents in many different like product service areas, uh, not just in what was traditionally an IDE, but also having things that are like based on triggers in your code base, in, in your repo at, you know, during CICD, uh, things that get triggered even when, you know, based on events, things like, you know, SRE style bots.”

Claim: The course’s eighth week covers background agents deployed beyond traditional IDEs, including agents triggered by repository events, CI/CD, and SRE-style tasks.

“Um, how do you even build a system like a... Uh, I think this past year was when a lot of companies started releasing their posts like Ramp Inspect. Um, there was Spotify has their own version of this. You know, I think Stripe Minion. So all these folks were doing really cool write-ups of like their internal infra they were building to do this. I wanna touch on that, uh, and get people exposed to what that is.”

Claim: The speaker says Ramp, Spotify, and Stripe have published accounts of internal infrastructure they built for background-agent systems.

“Yeah, this is, um, yeah, this one is, uh, more on like what happens when you're in, in a team in, in the modern world and sort of like the core pieces of, of team-level infrastructure. So everything from, uh, LM gateways to, you know, MCP portals, which is like MCP at scale, and then just also how to think about the evolving roles, I would say.”

Claim: The course addresses modern team-level infrastructure, including LLM gateways and MCP portals that support MCP at scale.

“So this is maybe a little bit like a soft thing. It's like what it, what, what is now... You know, and the, and the v- the vanilla maybe like not super nuanced thing to say is like everyone's a builder, but I wanna like go a little bit deeper beyond that and like sort of like what thing is actually getting subsumed? What, what roles are actually, um, either evolving or merging that didn't once exist? Um, what should students think about in terms of defensible skill set that's maybe wasn't true before?”

Claim: The course examines how roles are evolving or merging and what defensible skills students should develop.

“Um, and so a lot more like team dynamic type, type considerations. And then, yeah, this is, uh, it all kind of builds up toward the, toward the factory, which I think is top of mind for now, um, for a lot of folks today.”

Claim: The course builds toward software factories, a topic the speaker says is currently top of mind for many people.

“Once we're, we're building factories, where do humans still exist in all that? What is the, you know, where are the new choke points that are, that are intentionally there around, around human involvement?”

Claim: The course examines where humans remain involved in software factories and which bottlenecks are intentionally maintained around human participation.

“Things like intent, things like, uh, setting up a verifiable systems and what happens in the limit when maybe 89% of code is fully driven by just agents, uh, where we g- we kind of filled into all that.”

Claim: The speaker considers intent and verifiable systems in software factories and raises the possibility that, in a limiting scenario, perhaps 89% of code could be driven entirely by agents.

“Are we talking about these sort of half lights on factories? Like I think that even there's a spectrum there. Yeah, and there, you know, there's the people that are on the, on the very frontier, like, like the warps and the factories that are, that are like building factories and selling them as services and, you know, even they will only claim like 30 some percent, if even that, of, of like fully, you know, agentically driven contributions.”

Claim: People working at the frontier of software factories claim that roughly 30% or less of their contributions are fully agentically driven.

“... what does that mean in the limit? It's, it is hard to say.”

Claim: The endpoint or limit of agentically driven software factories is difficult to characterize.

“Um, just be aware that, you know, not everyone... They, they probably believe the things they're saying at the time- ... but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, "Yeah, I am often wrong."”

Claim: People probably believe their stated views when they express them, but they can and do change their minds.

“... but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, "Yeah, I am often wrong."”

Claim: Boris recently tweeted that he is often wrong.

“Like, and like everyone is like, because he created Cloud Code, is like treating his word as gospel.”

Claim: People treat Boris’s statements as authoritative because he created Cloud Code.

“And I'm like, "No, well, sometimes he disagrees with himself."”

Claim: Boris sometimes expresses views that conflict with his own other views.

“Yeah. Yeah. And, and that's okay. I mean, that's like this isn't politics, right? This is we're, we're still scientists. We're engineers. We're, we're subject to... I, I don't claim to be dogmatic, but you know, I just want people to experiment.”

Claim: The speaker says they are not dogmatic and encourages people to experiment.

“I think that's like a principle as well. You guys talked about principles earlier. Just like experimentation. I mean, like you just have to be ready to... A- and oftentimes when I, when I sort of like give sort of these lectures or, or talk to people about this stuff, they're like, "How do I...”

Claim: The speaker considers experimentation an important principle and says people should be prepared to experiment.

“There's so much going on. How do I stay on top of this?" And it's like, you know, there's something as simple as just like experimenting for 10 minutes a day. Like just spending some 10 minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new like framework or something, and just spending-- maybe even doing the exact same problem just with that new kind of thing is, is, is alone will cover most of the gap.”

Claim: The speaker says that experimenting for 10 minutes a day with a new tool or framework, even on a familiar problem, can by itself cover most of the gap in keeping up with changes.

“Pretty good, I would say. Like, you know, it's hard to benchmark these things. There is a review bench. No one uses it. Um, but like qualitatively, Devin review is pretty good.”

Claim: The speaker considers Devin Review qualitatively good but says these tools are difficult to benchmark.

“There is a review bench. No one uses it. Um, but like qualitatively, Devin review is pretty good.”

Claim: The speaker says that no one uses the review benchmark.

“I mean, like, yeah. So, so Mon- Monaco does use Devin. Uh, and that's, that's, uh... I mean, and Si- Silas actually spoke last year, and so Si- I mean, Silas is great. Just super, super smart. But, um, yeah, we, we use you guys, and so I was like, "Okay, among, among the providers of, of kind of code review tools that we..." And we sampled a lot of them. Um, we actually did kinda converge on Devin for- ... a lot of our use cases.”

Claim: Monaco uses Devin and, after sampling many code-review tools, selected Devin for many of its use cases.

“Yeah. Yeah. Yeah. So, so we do use agent readiness of our own, of our own kind. I mean, Eno is, is awesome as well. Like, I, I think they were super early on this, on this train, and I think when he codified that in his talk, I guess at AIE, what was it?”

Claim: The speaker praises Eno and says Eno was very early in developing agent readiness.

“Yeah, Europe or, um, just fantastic talk, and I think it was kind of in a way ahead of its time, and so I was like, "There's only one person that I could see being like the most- ... most appropriate person to talk about this," where I like really wanna give him the spotlight, and they're super excited.”

Claim: The speaker considered Eno’s talk on agent readiness to be ahead of its time.

“Oh, yeah. It was funny. Like I was at college, we were talking about this stuff and, like, we were like, "Okay, we need to form some opinions about it." And then, like Factory, like shipped a full product on it that week that we were talking about it.”

Claim: Factory shipped a complete agent-readiness product in the same week that the speaker and others were discussing the topic.

“Right, right, right, right, right, right. But you know, there's, there's value. And like I think they did well from a marketing perspective jumping on like the, the readiness report, and I've definitely spoken about... You know, I, I will regularly kinda pitch that to other people and, and say like, "Yeah, you should run readiness reports of your own kind." And, um, you know, of course now people have open source versions of it, but, but I think the, the principles are super important.”

Claim: The speaker considers the principles behind agent-readiness reports very important and regularly recommends that others run their own versions of these reports.

“Okay. So, so, so, uh, okay. Uh, this one is maybe a little bit more marketing-y, but, but okay. Um, the one thing I do wanna highlight a-and I've made that-- I also made that clear last year, but I'm not a, I'm not endorsing any tool, okay? Like, like I, I don't, I, I, I think that is, uh, the wrong way to approach something like this. Um, there are tools that come and go, and when... You know, transparently, when, when the c-class was taught last year, Claude Code was not as popular.”

Claim: The speaker does not endorse any particular AI coding tool and considers tool endorsement the wrong way to approach teaching AI-native engineering.

“Like, like I, I don't, I, I, I think that is, uh, the wrong way to approach something like this. Um, there are tools that come and go, and when... You know, transparently, when, when the c-class was taught last year, Claude Code was not as popular. It wasn't as like readily accepted. Boris was only at four thousand Twitter followers, and now he's at, you know, whatever million. I don't even know if he is, but you know, like he's at a lot of followers.”

Claim: Claude Code was less popular when the class was taught the previous year, illustrating that AI coding tools can change in popularity over time.

“Um, so, so Claude Code has really like, uh, emerged on the scene. Um, I think what-”

Claim: Claude Code has become prominent in the AI coding-tool landscape.

“Where there was suddenly this huge arms race around true like autonomy in these agents, like long-running agentic coding agent tasks. But yeah, so, so, like, you know, back to your point, so it's like less on...”

Claim: The speaker says that a major arms race developed around autonomous agents capable of long-running coding tasks.

“The TLDR was that, you know, you could sort of, through like pretty much introductory algorithms level understanding, you could just define like essentially an agentic loop that does the bulk of what you would call a coding agent.”

Claim: A basic coding agent that performs most core coding-agent functions can be built by defining an agentic loop, using knowledge at about the level of introductory algorithms.

“Um, and of course, there's like bells and whistles. There's no sub-agents. There's no MCP, none of that stuff. But if you wanna see something that you talk to and then it literally will edit your file, and you can do s- pretty sophisticated things with it, you c-- it doesn't require more than two hundred lines.”

Claim: A basic conversational coding agent that edits files and handles fairly sophisticated tasks can be implemented in no more than 200 lines of code, without features such as sub-agents or MCP.

“So I, you know, a lot of what I try to do in education is to try to like peel away the stuff that's like the secret sauce and show that like really at its core, I mean, these are still students, and you want to teach conceptual principles.”

Claim: The speaker's educational approach is to demystify AI tools by exposing their underlying mechanisms and teaching students the core conceptual principles.

“There's nothing more to this than... In Stanford, there's this class called 106B. It's like the introductory alg- intro to algorithms. You don't need more than 106B to, to build something like this. And so there you go. Um, there it is.”

Claim: Knowledge no more advanced than Stanford's 106B introductory algorithms course is sufficient to build a basic coding agent of the kind discussed.

“Right. Right, right. Exactly right. Yeah, yeah, yeah. Right. So yeah, it's, uh, it, it-- Not really. I think they're... I haven't got- received any like pushback on, on like this idea at all. I think they're actually more just...”

Claim: The speaker has not received student pushback against teaching the idea of building a coding agent from scratch.

“Most of them are consumers, right? Most of them, most of the students, uh, do just like end up using the tools. I don't think that they get a lot of exposure, um, if any, certainly in the coursework, around peeling that onion back, right?”

Claim: Most students use AI tools as consumers and receive little or no coursework exposure to understanding their underlying mechanisms.

“And I think, w- um, so this is the week of the harness. So, so, so Tuesday, first lecture, you know, logistics, blah, blah, blah, all that stuff. But then first half is fir- you know, the first true like bit of curriculum is the two hundred lines of, of, of like a, of a harness.”

Claim: In the course's week focused on coding harnesses, the first substantive curriculum lesson is a roughly 200-line harness.

“And then yesterday, when I had sort of the follow-up to that, is like we actually took Claude Code and then, you know, you have these like proxy things. You, these like pr- proxy tools where you can intercept the request. Yes, exactly. Exactly right. So use that. Uh, u-use that tool. Intercept the requests and responses. Look under the hood, see what's happening there, and then, then just like, okay, I did say you have Claude Code in two hundred lines, but really, okay, here's like actual Claude Code.”

Claim: The speaker taught students to use proxy tools to intercept Claude Code requests and responses and inspect the system's internal behavior.

“And so what I literally did is I took Opus 5.5, so this like, you know, model two days old, and was like, "This is... I intercepted those requests. Here's exactly what's going on. Here's the prompt. Here's the system prompt. Here's the plan prompt. Here's the compaction prompt."”

Claim: The speaker used Opus 5.5, described at the time as a two-day-old model, to analyze intercepted Claude Code requests and inspect its system, planning, and compaction prompts.

“And then we just like went through all of that and sort of showed it to the students. So now they really, really have like a st- a state-of-the-art harness that they can understand.”

Claim: The course walkthrough left students with a state-of-the-art coding harness whose operation they could understand.

“Yeah. I think it's a bit self-selecting- ... because the curriculum is all here, right?”

Claim: The speaker characterizes the course curriculum as self-selecting.

“So the curriculum has been available since, I mean, you know, definitely it's like a few weeks and, and, uh, I, I guess I announced it a few weeks ago, but it had kinda been in the works for like probably at least one to two months.”

Claim: The speaker says the curriculum was announced a few weeks earlier and had probably been in development for at least one to two months.

“So but if you looked for it, you could probably find it. Um, a-and I feel like the reality is if students are not interested in this, then they won't, like, they'll, they just won't show up and, you know, they, they will, they will choose not to show up.”

Claim: Students who are not interested in the curriculum will choose not to attend it.

“What I'm finding that is interesting is there are still definitely students that show up, but then there's also just a lot of folks from, you know, broader Stanford post-docs at other departments. There's a lot of like interdisciplinary- ... sort of interests, where folks that are in, you know, somewhere in pathology or like, you know, m- like medicinal sort of fields, or, or even like random other engineering fields that aren't pure CS that are coming in and are also interested in, in kind of getting exposed to this.”

Claim: The speaker reports that the curriculum attracts students as well as many Stanford postdocs from other departments and other interdisciplinary attendees.

“... sort of interests, where folks that are in, you know, somewhere in pathology or like, you know, m- like medicinal sort of fields, or, or even like random other engineering fields that aren't pure CS that are coming in and are also interested in, in kind of getting exposed to this.”

Claim: The curriculum draws attendees from pathology, medicinal fields, and engineering disciplines outside pure computer science who want exposure to the subject.

“A lot, also a lot of, uh, GSP students, MBAs, um, that also, uh, wanna I guess, learn how to not have a technical co-founder or whatever the- ... the motivation there is, but...”

Claim: The speaker says many GSP students and MBAs attend, possibly to learn how to build without a technical co-founder, while acknowledging uncertainty about their motivation.

“You'll be shocked. Uh, Claude Code has the longest system prompt of all the coding agents, something like twenty thousand tokens.”

Claim: The speaker says Claude Code had the longest system prompt among coding agents, at approximately 20,000 tokens.

“Yeah, I know, but then they grew again, and then they cut and they grew again.”

Claim: The speaker says Claude Code's system prompt has repeatedly been reduced and expanded in size.

“So in the early days, they were among the first to really lean into this, "Let's fewer tools do more," right? Lean into SQL, lean into Bash, lean into, like, the things that are, that are powerful, like Swiss Army knife tools, and have that be a lot of the driver of capability.”

Claim: The speaker says Claude Code initially emphasized simplicity, relying on a few versatile tools such as SQL and Bash to provide much of its capability.

“Um, I think they did a fantastic job leaning into that. I think that's why they, they were as popular as they did. I think that when I presented something like the system prompt, it was like, well, there was a different system prompt about two months ago.”

Claim: The speaker believes Claude Code's emphasis on simplicity and powerful tools contributed to its popularity.

“It is now-- They have reduced in complexity, and it turns out that now what was once part of the system prompt, things like how to use even Git, right? Or how to, like, set up tasks or things of that nature, are no longer part of the system prompt.”

Claim: The speaker says Claude Code's system prompt has become less complex and no longer contains instructions such as how to use Git or set up tasks.

“It's actually expected to be more kinda captured in the models, like the model weights itself. Um, so they've kind of baked in a lot of those learnings more at the model level. And that just raises interesting questions around, you know, the model versus the harness, right?”

Claim: The speaker says that knowledge previously included in Claude Code's system prompt, such as tool-use instructions, is now expected to be encoded more in the model weights.

“And how do you kind of think through that trade-off? What do companies invest in? What should you invest in to really push the, the frontier of your own capabilities? And so I, I think we still, even as a, as a community, we seesaw between the two a lot.”

Claim: The speaker says the AI community continues to shift back and forth between investing in model capabilities and investing in the harness to advance capabilities.

“At some level, it's also how the model was, right? Uh, weaker, older models, you need to give them examples of, "This is good, this is bad," and then obviously you cut that away at some point, right?”

Claim: The speaker says weaker, older models may need examples of good and bad behavior in their prompts, and those examples can be removed later.

“You can apply the same towards if you're using a local coding agent with not the strongest model, you might need to add these examples, right?”

Claim: A local coding agent using a less capable model might need examples added to its prompt.

“That's right. That's exactly right. Yeah. Yeah. And then, and then of course, there's, uh, interesting AI learnings to follow out of that, which is why, you know, as the models get better, when are you over-constraining it by adding those examples, right? ... pull away the over-constraining? Uh, and that's, I think, part of the learning as well.”

Claim: The speaker says that as models improve, examples may over-constrain them, so learning when to remove those constraints is part of the process.

“Well, no, it's, it's also a trailer, right? It's like, well, if you want more, go to the source. If you're signing up next, next year or next fall for courses, you know, this is your one hour. And you know the best part? If it's a trailer, by the next time you teach it, it'll change again.”

Claim: The speaker describes the one-hour version as a trailer for the full course and says the course will likely change by its next offering.

“That's true. That's true. That's true. That's true. Yeah. Well, yeah, so, so, okay. So a good call out on industry. It's, it's funny. Um, the-- I think that the, the main thing that I just wanna reinforce here is, uh, and I see this even when I'm, like, interviewing folks, uh, the very few people in industry even progress beyond, like, just vanilla prompting, right?”

Claim: Only a small number of people in industry move beyond basic prompting when using AI models.

“There's like a vanilla form, like, "Hey, do me the thing," right? Like, "Do, do the thing." And, and that certainly as the models get good, that can carry you a certain distance.”

Claim: As AI models improve, basic prompting can be useful to a limited extent.

“But inevitably, you know, any folks, folks I speak to that are-- come out of, like, a Google or some place where there's a massive, like, code base, the reality is you have to be a lot more thoughtful about how you're, like, ingesting context, right?”

Claim: Working with very large codebases requires more deliberate methods for providing context to AI systems.

“And so thinking through what are some of those principles and those techniques around how to, like, point, you know... And I sort of like, you know, the Repit framework that I'm sort of describing here is, is functionally similar to, like, Dex's RPI, right?”

Claim: The speaker's Repit framework is functionally similar to Dex's RPI framework.

“Or something of that nature, where you have this, like, re-research, kind of proposal, and sort of s- a spec, and then you, and then you do, like, uh, your implementation, and then you kind of tie it together with some testing and whatnot.”

Claim: The described workflow proceeds through research, a proposal or specification, implementation, and testing.

“So what I just wanted to reinforce here is that, like, you should have this step, especially when you're entering unfamiliar code bases, when you're trying to, uh, sort of infuse into the context something that an agent needs to understand about a very large code base.”

Claim: When working in an unfamiliar, very large codebase, developers should include a research phase to help provide the agent with the context it needs.

“There's value in something that is a research phase. And then in the case of this particular sort of framework that I'm putting forward, it's like also then suggests some proposal where there's a few alternative ways of solving a problem once you've done the research.”

Claim: The speaker's framework recommends generating several alternative solutions after researching the problem.

“And so this is where I sort of also invite the human in the loop to the sort of be the decision maker on one appro-- one approach versus the other. And then you enter the plan, which is kind of the spec.”

Claim: In the speaker's workflow, a human chooses among proposed approaches before the process moves to planning and specification.

“All to say that the, the TLDR is, like, there is a way to do this that actually does scale out to code bases that are, you know, millions of lines long. Um, and it's not just the vanilla prompt.”

Claim: A more involved workflow than basic prompting can scale to codebases containing millions of lines.

“Uh, and I think that's what I wanna highlight. If you're gonna go work in these big code bases, you should use something a little bit more, little bit more involved.”

Claim: Developers working in large codebases should use a more structured approach than a basic prompt.

“I think that probably the weirdest thing about how I think about context right now is when you split out to a sub-agent. Because sub-agents can benefit from inheriting your entire context of the main thread or not. Uh, sometimes, uh, there's a regular disagreement on, on that, and then sometimes you want to condense what the sub-agent does, and researchers put it back onto the main thread.”

Claim: Whether a sub-agent should inherit the full context of the main thread depends on the situation.

“Because sub-agents can benefit from inheriting your entire context of the main thread or not. Uh, sometimes, uh, there's a regular disagreement on, on that, and then sometimes you want to condense what the sub-agent does, and researchers put it back onto the main thread.”

Claim: In some cases, a sub-agent's work should be condensed before being returned to the main thread.

“And it's very hacky. I don't like anything I'm doing. So I don't have any answers. All I know is, like, uh, when you want to fan out work-”

Claim: The speaker considers current approaches to managing context for sub-agents hacky and says they do not have a solution.

“... uh, it is not just about you have a big spec and you, you, you know, you do RPI, you do R- Repit, whatever. It is not one agent anymore.”

Claim: Distributing work across agents involves more than writing a large specification and applying RPI or Repit; it requires multiple agents.

“It is about fanning out work and making sure they all have the right context, and they all have the right guidance. If you have a review agent reviewing all the other works, you have to make that guy have, have everything.”

Claim: Effective multi-agent work requires distributing tasks while giving each agent appropriate context and guidance.

“It is about fanning out work and making sure they all have the right context, and they all have the right guidance. If you have a review agent reviewing all the other works, you have to make that guy have, have everything.”

Claim: A review agent overseeing other agents' work needs access to the full relevant context.

“... "Oh my God, these guys don't have enough context. I'm gonna give them more context." Okay, great. And then they're gonna overtune to some random one-off comment that you made t- a while ago, and then make some really stupid mistakes 'cause you, you didn't mean it that way. But like, they took it as the law.”

Claim: Providing agents with excessive or poorly contextualized information can lead them to treat an incidental comment as a rule and make mistakes.

“Something like that. And so I don't know what, what the word is I'm saying, right? It's just like multi-agent management for context.”

Claim: The speaker characterizes the challenge as managing context across multiple agents.

“It's like some combination, I think to your point. Yes, the fan-out for the exploration, the fan-out for like the subtasks and then, and then the consolidation, compaction, whatever, you know, you might wanna call it.”

Claim: Multi-agent context management involves distributing exploration and subtasks among agents, then consolidating or compacting their results.

“I think being thoughtful about refreshing of the context, understanding when things are getting a little bit too bloated. I think, you know, Dex had this concept of like the dumb zone and sort of like being- ... a little bit more thoughtful- ... about the dumb zone.”

Claim: The speaker invokes Dex's concept of the "dumb zone" as relevant to managing context thoughtfully.

“I think being thoughtful about refreshing of the context, understanding when things are getting a little bit too bloated. I think, you know, Dex had this concept of like the dumb zone and sort of like being-”

Claim: Context management should include deliberate refreshes and recognition of when the context has become excessively large.

“Uh, yeah. Well, I think he's-- I mean, he helped-- definitely informed a lot of the, the curriculum here, and so yeah, he's just fantastic, I think, thinker and all this. Um, no, I think the diagrams are, are huge. That's useful.”

Claim: The speaker considers diagrams highly useful.

“Uh, yeah. Well, I think he's-- I mean, he helped-- definitely informed a lot of the, the curriculum here, and so yeah, he's just fantastic, I think, thinker and all this. Um, no, I think the diagrams are, are huge. That's useful.”

Claim: Dex helped shape much of the curriculum, and the speaker regards him as a fantastic thinker.

“Cloud-managed agents has a version where sessions are separate and it's not inherited. This used to be a big talk- topic before reasoning models-”

Claim: One version of cloud-managed agents uses separate sessions without inherited context.

“... when agents didn't run as long. Um, we had shorter context windows. Everyone was very, very managing that precious context window, right?”

Claim: Before reasoning models, agents ran for shorter periods, and users closely managed their more limited context windows.

“Uh, Link Chain used to write a lot of blogs about this in good context engineering. All this stuff is good for people to know. I think in the student context as well, you can expect the average student to take your class 'cause it's a great class, and they'll know all these things, so you should know them, know them better as well.”

Claim: Link Chain previously published many blog posts about context engineering.

“Uh, Link Chain used to write a lot of blogs about this in good context engineering. All this stuff is good for people to know. I think in the student context as well, you can expect the average student to take your class 'cause it's a great class, and they'll know all these things, so you should know them, know them better as well.”

Claim: Instructors should understand context-engineering concepts especially well because students in their class may already know them.

“Um, this is a fun space for research as well.”

Claim: The speaker considers context engineering an enjoyable area for research.

“One of the papers, research papers presented at AIE, they looked at the delta in the cloud code system prompt, what got changed, and they like, um, they, they plotted a whole bunch of changes in performance.”

Claim: A paper presented at AIE analyzed changes to the Cloud Code system prompt and plotted how performance changed alongside them.

“They tried to reverse engineer it. It's just good research that you can do. There was another one that also looked at context window management and how it affects performance downstream.”

Claim: Researchers attempted to reverse engineer the Cloud Code system prompt, while another study investigated the downstream performance effects of context-window management.

“They weren't the best papers, but like it's quite accessible, and it puts out good blog posts, so people that are trying to work on it, you know, I just-”

Claim: Although the papers were not especially strong, the research is accessible and yields useful blog posts.

“... recommend it. It's a, it's a good area you can easily poke into.”

Claim: The speaker recommends context-engineering research as an accessible area for people to explore.

“Understand the code, exactly. Yeah, you gotta understand the code and, and I don't know if g- you guys have used that, but effectively, like you run, you know, you have your, your, your kind of the, the, the diff you've put in your code, and then you run the skill, and at the end of it, you have this not only like kind of a visual representation of what's happened, so you can, you know, kind of skim it and understand what's, what's, what's actually at the core of it.”

Claim: The Explain Diff skill presents a visual representation of code changes so users can skim and understand the changes’ core content.

“But then what I really liked was this almost like quiz at the end of, uh, at the end of it, where you go through and sort of like really reinforce- ... whether you understand what's happening. And I think that, that level of interactive sort of back and forth I think is, is quite nice.”

Claim: Explain Diff ends with a quiz intended to reinforce users’ understanding of what is happening in the code changes.

“... whether you understand what's happening. And I think that, that level of interactive sort of back and forth I think is, is quite nice.”

Claim: The speaker considers Explain Diff’s interactive back-and-forth useful or appealing.

“I think there's a lot more output than people are used to. Like, I can type in a one-line prompt and get tens of thousands of lines of code and a whole structure, and-”

Claim: The speaker says that a one-line prompt can produce tens of thousands of lines of code and a complete code structure.

“Um, but you know, more effectively, you should know what you're working on.”

Claim: People using coding tools should understand what they are working on.

“You can really be put out of touch, out of the loop. You should spend more time thinking about what you're actually doing.”

Claim: People using coding agents can lose touch with and awareness of their work, so they should spend more time thinking about what they are doing.

“Uh, doing the work and getting the code written is much easier than it was.”

Claim: Writing code and completing coding work are much easier than they were previously.

“Yeah. So I mean, effectively, like skills that interact with, uh- ... with like the web itself. Yeah. Uh, I mean, mainly things that involve like pull and pull information from the web. I think it's, it's in that domain.”

Claim: Web skills interact with the web, including by retrieving information from it.

“Uh, the, this is, um, most I think students have like interacted with cloud code at this point, and so-”

Claim: By the time of the conversation, the speaker believes most of the students had used Claude Code.

“Yeah. Uh, I would say, you know, I think there's also another thing about skill. There are bu- skills that are bundles of skills- ... where they, they sort of cross-reference each other.”

Claim: Some AI-agent skills are bundles whose component skills cross-reference one another.

“Uh, like plugins, and then there's also, I guess, the art of writing a skill, which is a thing that I've also focused on for AI.”

Claim: The speaker has focused on the craft of writing skills for AI.

“Th- I, I think a lot of these, uh, are focused on skill-consuming. Um, and you have all these influencers like Matt Pocock and what have you, that are like, "Yeah, this skill will save you- ... will solve all your problems." But no, you should write your skills yourself.”

Claim: The speaker says much discussion emphasizes using pre-made AI skills and influencers may promote them as solving every problem, but people should write their own skills.

“And actually, when you try to write them naively, they're usually bad. And actually, it takes a little bit of skill to write a skill.”

Claim: AI skills written without care or expertise are usually poor.

“And actually, when you try to write them naively, they're usually bad. And actually, it takes a little bit of skill to write a skill.”

Claim: Writing an effective AI skill requires skill.

“So I, I, I... like, there-- So every, like, uh, Cloud Code and Codex, they all have skill creation skills, right? So then there's-- I have a skill cutter skill that cuts it down because they, they're all-- They, it just documents every way in which those default skills that are created are bad.”

Claim: The speaker uses a skill-cutter skill to shorten skills generated by Cloud Code and Codex, which the speaker says document their flaws excessively.

“So you always run this after any skill is created- ... to cut it down.”

Claim: The speaker runs a skill-cutter skill after creating any skill to shorten it.

“Uh, yeah. It's really annoying because sometimes there is slop and, like, but there's good in the slop. And then you, you have to, like, sort of whittle down what they're, what that is.”

Claim: The speaker says generated skill content can contain both undesirable slop and useful material, which must be distilled.

“It's fun. Everyone has their own version. I use-- First, there's very easy to, like, vibe check a skill, right? You just call it, run it a few times- ... and then I distill these preferences into a skill that's like, "This is what I like, this is what he doesn't like."”

Claim: The speaker evaluates a skill by running it several times and then encodes personal preferences into a skill.

“And it, it adjusts over time, and then, you know, it can automate a bit of it.”

Claim: The speaker says a preference-based skill changes over time and can automate part of the work.

“Yeah, well, so the, the problem is that y- yeah, you're calling it, but the, the actual-- The worst part is when a skill triggers when you're not expecting it to. Like, you wrote it one time to fix a thing- And then, like, now you're working on a completely different thing, and it triggers.”

Claim: The speaker considers unexpected skill activation during unrelated work to be a major problem.

“Why? 'Cause it's badly written. Because the trigger says, like, "Oh, call me whenever blah, blah, blah happens."”

Claim: The speaker attributes unexpected skill activation to poorly written, overly broad trigger instructions.

“You know, there's a, there's, like, a few people that are just like, "Okay, I have a bunch of skills I keep adding on sometimes when I start." And I know some people, like, uh, Eugene Yan, they are pretty particular about how many skills- ... they have. They prune them. If not used, they clean them every few weeks.”

Claim: The speaker says some users continually add skills, whereas Eugene Yan and others limit their skill collections and remove unused skills every few weeks.

“So, uh, so yeah. Th- this is, like, a, a huge problem if you're, if you're sort of in that skills world. I think this is actually why I suspect, uh, there are good-- there are very smart people at OpenAI and, and Anthopic that want skills to die.”

Claim: The speaker considers skill management and activation problems a major issue for people who use skills extensively.

“So, uh, so yeah. Th- this is, like, a, a huge problem if you're, if you're sort of in that skills world. I think this is actually why I suspect, uh, there are good-- there are very smart people at OpenAI and, and Anthopic that want skills to die.”

Claim: The speaker suspects that some highly capable people at OpenAI and Anthropic want the skills approach to disappear.

“But it just feels sloppy. This feels sloppy. I, I know it's... I, I can see my own skills repo being sloppy. It doesn't feel, uh, AI-pilled.”

Claim: The speaker considers the skills approach, including his own skills repository, sloppy.

“It's, it's not, it's not what AGI was supposed to look like. Uh, it just-- It's, like, everyone having a giant bunch of markdown files.”

Claim: The speaker says a future of people managing large collections of Markdown files is not what AGI was supposed to look like.

“A lot of people actually don't look at their markdown files, right? Like, the- ... the UX of it, of just chatting what you want, creating a skill in chat, using it, seeing output. There's a lot you can-- There's a lot of value you can get without ever looking at your markdown.”

Claim: The speaker says many users do not inspect skill Markdown files and can still get substantial value through a chat interface for creating and using skills.

“Uh, you can get a lot of value. If you do stuff repeatedly, just be like, "Look at the last five times we did this. Make me a skill," and you can iterate it with chat. There's a good side of it as well, right?”

Claim: The speaker says users can create and refine a skill through chat by asking it to derive one from the last five repetitions of a task.

“But then in the case of something like Devin, right, where we've codified skills as a way of, of sort of, of representing our best practices, right? And then making-- and exposing that to the agent or the code review system to then, you know, institute the APIs the way that we like to define them, institute kind of the service level boundaries the way we define them.”

Claim: The speaker describes organizational skills as a way to encode best practices for agents or code-review systems, including preferred API conventions and service-level boundaries.

“That's like an alternative way of sort of representing, like, org-level knowledge, and maybe not so much this, like, arbitrary thing you want the agent to do so you don't have to write the prompt each time.”

Claim: The speaker distinguishes skills that encode organizational knowledge from ad hoc procedures used to avoid rewriting prompts.

“Uh, and so I like y- I like that. I feel like that positioning might-- that, that purpose might exist longer than just, uh, you know, these kind of pr- random procedures.”

Claim: The speaker likes skills positioned as organizational knowledge and thinks that purpose might endure longer than ad hoc procedures.

“Well, here's the thing. Okay. On this, we, we, uh, we're about to buy... No, we're not about to buy the country. But, but, but, um, you know, we, we did, uh, we did get into a little bit of flack when we, uh, bought the monaco.com domain. I will say that, uh, we did get, uh, sued by them, so... Yeah. There's, there's like a whole public, there's like a whole public thing about this where like... Yeah, so they didn't like that, but, but here we are and, uh, we won. We won, so...”

Claim: The speaker says Monaco bought the monaco.com domain, was sued by the country or its representatives, and won the dispute.

“Because it's like a victory. It's because it's a victory. It's our victory. That's funny.”

Claim: The speaker says the flags displayed around San Francisco mark Monaco's victory in the domain dispute.

“Sam Blon came up with it, six thousand dollars a day. Uh, I don't necessarily think it directly causes me to google what Monaco is, but the-- all of G- Monaco's ads are just Monaco dollar sign.”

Claim: The speaker is unsure that Monaco's plane-based advertising directly causes people to search for Monaco online.

“There you go. Okay. That-- This is Monaco. This is the Monaco that, that I know and love. Uh, yeah. So we are, uh, you know, we're, we're a Series B company, uh, here based in SF.”

Claim: Monaco is a Series B company based in San Francisco.

“We are, are building effectively, like, an all-in-one sales platform. Uh, fully AI native agents doing all the fun stuff, uh, or at least the stuff that humans don't like to do, but, but that agents are-- do like to do 'cause they don't really have opinions, but do in a less error-prone fashion.”

Claim: Monaco is building an all-in-one sales platform in which AI-native agents handle tasks humans dislike, and perform them with fewer errors.

“So the salespeople and sales teams can focus on doing what humans are good at, which is, you know, closing deals.”

Claim: Monaco's sales platform is intended to let salespeople and sales teams focus on closing deals.

“Well, okay. So, so right now we are, uh, we started the, the company in a sort of like, we're-- The, the market that we went after was not enterprise from the get-go, right? We sort of said, like, we're gonna start by... Like, if our goal is to build the all-in-one, then we need to kind of build this in an iterative fashion with, with a, a group of customers, and we need to, like, grow with them into this. It's much harder to pull someone away from, from Salesforce than to say, like, "Hey, if we meet you at the point when you are still just a Google spreadsheet tracking pipeline, and we sort of show you a, you know, more AI-pilled sort of answer to the problem, then we can grow with you."”

Claim: Monaco initially avoided targeting enterprise customers, instead developing its all-in-one sales product iteratively with customers and growing with them.

“Like, if our goal is to build the all-in-one, then we need to kind of build this in an iterative fashion with, with a, a group of customers, and we need to, like, grow with them into this. It's much harder to pull someone away from, from Salesforce than to say, like, "Hey, if we meet you at the point when you are still just a Google spreadsheet tracking pipeline, and we sort of show you a, you know, more AI-pilled sort of answer to the problem, then we can grow with you."”

Claim: Monaco aims to win customers who currently track their sales pipeline in Google spreadsheets by offering them a more AI-oriented solution.

“And in a way, it kind of follows the same, the same, uh, model of like, you know, Brex or all the, like, YC companies, where you kinda sell to your batch mates, you kinda grow with them, and, and you- then you go up, up, up market.”

Claim: Monaco's go-to-market strategy resembles Brex and other Y Combinator companies: begin by selling to startup peers, grow with those customers, and later move upmarket.

“And so we are, I would argue, are the, you know, the, the... I would, I would claim the, the among the best, if not the best, sort of like, uh, startup-focused, uh, sales platform.”

Claim: The speaker claims Monaco is among the best, if not the best, sales platforms focused on startups.

“And so all of our customers are, like, early-stage startups. At this point, probably through to, like, Series A is, is kind of where, where we've primarily been focused, kind of seed to Series A.”

Claim: Monaco has primarily focused on early-stage startup customers, from seed stage through Series A.

“And then now we're going more up market, and we've kind of learned a lot about the all-in-one in the last year and a half of doing that.”

Claim: Monaco is now moving upmarket and says it has learned a lot about building an all-in-one sales platform over the past year and a half.

“Yeah. I mean, uh, we have built our own agentic harness for, for sales use cases. So, uh, all this, I mean, to, to call back to the, to the coding harnesses, uh, taking a lot of inspiration from, from the systems like the Codexes and the, and the Clods of the world, and sort of building bespoke tools that are very sales-focused, bespoke workflows that are all kind of geared toward GTM use cases.”

Claim: Monaco built a bespoke agentic harness with sales-focused tools and workflows for go-to-market use cases, drawing inspiration from coding systems such as Codex and Claude.

“Um, so we built our own version of that, that powers pretty much every single thing, that, that powers the Monaco platform. So from, you know, uh, scoring of, scoring of TAMS, uh, scoring of ICPs and, and develop- devo- defining of ICPs- ... uh, through to pipeline evolution, through to, like, all the mundane kind of things that maybe a salesperson doesn't wanna do, which is like, oh, update my meeting, you know, pre-meeting notes, update my summaries, like all these things.”

Claim: Monaco's agentic harness powers nearly all of its platform, supporting market and ideal-customer-profile scoring and definition, pipeline evolution, and routine sales tasks such as updating meeting notes and summaries.

“Um, you know, we have agents powered, powering all the, those workflows. And so we- we've done that. It, it's, it kind of comes, comes out of the box. Um, and, uh, yeah, so long, long agents, long-horizon agents for... long-horizon agents and short-horizon agents in some cases for, for, for GTM use cases.”

Claim: Monaco's go-to-market workflows use both long-horizon agents and, in some cases, short-horizon agents.

“Yeah, it's a good call-out, uh, because so far... I mean, I think this is generally true of a lot of, like, vertical AI companies, is like you kinda start where you sort of use the frontier models, like, as third-party services to see what's possible, 'cause they are, like, the very frontier.”

Claim: The speaker believes vertical AI companies often start by using frontier models as third-party services to test what is possible.

“... like the fif- plus 50% to minus 50% margin kind of question. I mean, you know, I think ir- irrespective of the numbers, I think what is interesting is, like, at a certain point, things do become interesting from a cost perspective.”

Claim: The speaker believes that model choices eventually become significant from a cost perspective.

“You, you start to wonder, can you do it faster on o- you know, open weight models? And then that's a lot of that... Then a lot of what I think the, the purpose of, of the AI pieces are is just to post-train those models, build environments for, you know, kind of to post-train using RL and things of that nature.”

Claim: The speaker sees evaluating open-weight models for faster execution and post-training them with reinforcement learning in purpose-built environments as potential work for AI teams.

“So, um, we're kind of in that phase right now.”

Claim: Monaco is currently exploring open-weight models and post-training approaches.

“No, no, no, no. Um, but yeah, we use Parallel. We also use Exa. Um, so we'll give, give a shout to both of those folks. Um- They're both friends. They're both folks that, uh, that we like to work with.”

Claim: Monaco uses Parallel and Exa for enrichment and the speaker says the company likes working with both, without expressing a preference between them.

“I'll keep it, I'll keep it, uh, neutral. I'll stay, I'll stay Switzerland in this today. But yeah, no. So, so and then, of course, we do our own version of enrichment as well. So we-- it's, it's like a-- most of our data systems, I mean, part of like one of, one I think of the most valuable pieces of software that we build or most valuable systems is currently like our world database, like the database of companies, enriched companies, enriched accounts, uh, enriched contacts.”

Claim: Monaco builds its own enrichment, and its database of enriched companies, accounts, and contacts is among the company's most valuable software systems.

“So like that whenever we are constructing ICPs for folks, we have the most up-to-date information about who you should reach out to at which company at this point in time.”

Claim: Monaco uses its database to provide current recommendations about which people to contact at each company when constructing customer ideal customer profiles.

“Like, I would say that one of our-- like I think where we are like world-class, hands down, is just demand. Like we are a very good demand company. Like we help companies increase their meetings booked in a way that actually-”

Claim: The speaker says Monaco is particularly strong at demand generation and helps companies book more meetings.

“If they are now in the middle of like a job, like a, a good signal is like once-- if they've gone from one job to a new job, they're now like entering a new place where they're more likely to be-- to b- purchase or be open to a purchase of a new sales platform because they're in the middle of that job transition.”

Claim: A VP of sales who has recently changed jobs may be more likely to consider purchasing a sales platform, making the job transition a useful time for outreach.

“So going from company A to company B, for like the VP of sales, a good signal of like, "Hey, should you reach out to them at this point in time?" is did you detect that their LinkedIn changed or something like that.”

Claim: Detecting a VP of sales's LinkedIn profile change can signal that the person may be at a good time for sales outreach.

“Yeah. Yeah. No, I mean, we have-- So we, we made investments in coding agents from basically like day one, and we've like learned a lot. We have our own like scars that we've sort of like picked up and, and learnings.”

Claim: Monaco has invested in coding agents since its early days and has accumulated lessons from that work.

“And I think we've now been getting much deeper into sort, sort of sharing some of our learnings. So like in our blog, which might be worth sharing here, we've been increasingly posting some of what, uh, maybe not the...”

Claim: Monaco has increasingly published some of its coding-agent learnings on its blog.

“So you know, we spent a lot of time looking at different providers, looking at vendors, and then ended up building a lot of things in-house. So for us, something like a background agent platform, we have our own version of like a ramp inspect, um, that we call Monocoder- ... um, which is, is built on the Coder platform, um, but it is what we use to essentially scale up our engineering productivity.”

Claim: Monaco's background-agent platform, Monocoder, is built on Coder and is used to increase engineering productivity.

“So you know, we spent a lot of time looking at different providers, looking at vendors, and then ended up building a lot of things in-house. So for us, something like a background agent platform, we have our own version of like a ramp inspect, um, that we call Monocoder- ... um, which is, is built on the Coder platform, um, but it is what we use to essentially scale up our engineering productivity.”

Claim: After evaluating different providers and vendors, Monaco chose to build many of its tools and systems in-house.

“We actually, yeah, have sort of adapted it to some of our use cases. And yeah, it's where a lot of our agent development happens. It's just this really well-designed piece of software for essentially isolating of, of sort of like, um, dominant environments, scaling out to multi-agents, um, you know, being able to run these like long, long horizon agentic coding agent tasks.”

Claim: Monaco adapted Coder's software for some of its use cases, and uses it as a major environment for agent development.

“We actually, yeah, have sort of adapted it to some of our use cases. And yeah, it's where a lot of our agent development happens. It's just this really well-designed piece of software for essentially isolating of, of sort of like, um, dominant environments, scaling out to multi-agents, um, you know, being able to run these like long, long horizon agentic coding agent tasks.”

Claim: The software Monaco uses supports isolated environments, multi-agent scaling, and long-horizon agentic coding tasks.

“So Monocoder, shout out to John Tracy and, and, and co for, for building that out. Um- And, and then, yeah, like, from there we've also built our own versions of like a, you know, like a, like a Slack bot, which is-- we call Hal.”

Claim: Monaco developed an internal Slack bot named Hal.

“But, um, you know, effectively Hal is like this part AISRE, part like customer success, part like, uh, kind of QA engineer because it, it, it lives in our Slack, but it, it serves all the purposes of helping us debug things and put up PRs when things are broken, but also used by our own salespeople to find out how the product works, find out, like, how to get to the bottom of something.”

Claim: Monaco's Slack bot Hal functions as an AI SRE, customer-success assistant, and QA engineer, helping debug issues, create pull requests for problems, and help salespeople understand the product.

“And so there's, like, interesting questions we've had to answer around the different personalities that Hal has, uh, depending on whether it's talking to, like, a salesperson versus a, versus like a, a develop- developer.”

Claim: Monaco has had to design different conversational personalities for Hal depending on whether it is interacting with salespeople or developers.

“And so systems like this that we built out over time a lot in the last, like, year or so. Uh, shout out to Yunja for, for Hal, uh, and, and John Tracy as well.”

Claim: Monaco developed systems such as its agent-development tools and Hal over time, with much of that work occurring in approximately the prior year.

“We do use other tools, yeah.”

Claim: Monaco uses some third-party tools in addition to its in-house systems.

“Which is like you build to learn and you buy to keep. Or sorry, you buy to learn and you build to keep. Yeah, the other way around there. So, so you, you buy to learn and you build to keep, and I think that's generally true here, where in the early days when we were still trying to figure out, like, you know, get our feet under us a little bit, we were, you know, CodeReview as an example.”

Claim: The speaker’s general principle for choosing tools is to buy tools to learn from them and build tools in-house to keep using them.

“One of them where we spent a lot of time looking at different code review platforms. It turns out today we still use Devin because we've actually still find a lot of value in how the system's been built. So, you know, shout out to the cognition team. Um, but then along the way, we, we did try out some like SRE tools and, you know, we tried some sort of like off the shelf, maybe more like background agent platforms.”

Claim: The speaker’s team continues to use Devin for code review because it finds value in the way Devin is built.

“Decided that it was just, given our infrastructure, given how we had set up our own cloud platform, it just made more sense to kind of build this out internally. Of course, we're not dogmatic. We, we are open to revisiting them if, if something comes out.”

Claim: Because of its infrastructure and cloud platform, the speaker’s team chose to build some tools internally, but is open to reconsidering if a suitable offering emerges.

“But here I felt-- I think usually we ended up building in-house when we felt that there was no, like, mature offering out there. So yeah, and CodeReview is maybe a interesting example because we did do a whole survey of CodeReview tools, like maybe over a year ago.”

Claim: The speaker’s team generally built tools internally when it judged that no mature external offering was available.

“Graphite. Yeah, Graphite for a while before they got acquired. Um, like Ellipsis and, uh, which like YC company before the fact. So but there's a bunch, right? I mean, so, so there were a lot of folks that we did try and they, they were okay, but w-- I, I don't think they have like crossed the threshold of like they're immediately useful yet.”

Claim: In a code-review-tool evaluation more than a year ago, the speaker judged that the products tested were not yet immediately useful enough.

“There was... Yeah. This, this was like... I, I mean, okay, okay. Here, here's what I'll say. When we did that first evaluation the first time around where we didn't really pick any vendor, um, and we were like, "There's no vendor that we're super happy with," I did do like an analysis where I looked at our PRs. Basically did sort of like a, you know, set of pie diagrams, like here's what this is solving. Are we, are we happy with the kind of the trade-off of verbosity and how often it's posting- -with utility?”

Claim: In the first code-review-tool evaluation, the speaker analyzed pull requests to assess what the tools addressed and whether their verbosity and posting frequency were worthwhile given their utility.

“Um, and so there was-- it was more statistically driven than just vibes. The second time around, I wasn't gonna push hard on it. The first time I was kind of like, "Hey, we should adopt an AI code tool." Second time I was like, "No, I'm not gonna do that again."”

Claim: During the second evaluation, the speaker chose not to strongly advocate for adopting an AI code tool.

“Um, and so there was-- it was more statistically driven than just vibes. The second time around, I wasn't gonna push hard on it. The first time I was kind of like, "Hey, we should adopt an AI code tool." Second time I was like, "No, I'm not gonna do that again."”

Claim: The speaker’s first evaluation of AI code tools used more statistical analysis than subjective impressions alone.

“Um, but then organically people were just like, "Hey, Devin is like good. Like, can, can you like add me to this?" And so today people are sort of just like getting organically added. So this is a vendor where, you know, one of the cases where we're still leaning to the vendor.”

Claim: Colleagues who considered Devin good began requesting access, and the speaker’s organization now adds users to it organically.

“Um, but then organically people were just like, "Hey, Devin is like good. Like, can, can you like add me to this?" And so today people are sort of just like getting organically added. So this is a vendor where, you know, one of the cases where we're still leaning to the vendor. You know, we'll see in the long term what that looks like. But, um, everything outside of that, we just didn't feel like the maturity was there.”

Claim: The speaker’s team currently leans toward using Devin as a vendor, though the long-term decision is uncertain.

“You know, we'll see in the long term what that looks like. But, um, everything outside of that, we just didn't feel like the maturity was there.”

Claim: The speaker’s team judged that the other tools it considered were not mature enough.

“I, I think that the main piece that I want to, uh, reinforce, something like agent-ready code, code bases is to me like a precursor and a bit of a building block toward everything to come afterwards.”

Claim: Agent-ready codebases are a precursor and building block for later capabilities, including multi-agent systems.

“When I say everything to come afterwards, I mean like people always got, you know, for a while, and I think people still are like super hyped on like multi-agents, like paralyzing agents, like tens of agents doing all kinds of things.”

Claim: People have been and may still be highly enthusiastic about multi-agent systems in which many agents work in parallel on different tasks.

“It's like you actually need to set a foundation in order to do that well, and the foundation is how verifiable is your code base, how like well-structured is your code base, how understandable is it by an agent?”

Claim: Multi-agent systems require a foundation of codebases that are verifiable, well-structured, and understandable to agents.

“And so I think like agent readiness to me is like almost, almost like piece one of the next four to five weeks where maybe until like week eight or nine where, where I spend more time talking about like broadly, uh, you know, AI software teams and like kind of org level structure.”

Claim: In the course, agent readiness is an early topic, preceding later instruction on AI software teams and organizational structure around weeks eight or nine.

“But definitely building up the, the building blocks for, for, uh, background agents, uh, the software factory. I think this is where it all kinda gets interesting from, from that perspective.”

Claim: The course builds foundational concepts for background agents and the software factory.

“That's right, that's right. Yeah, um, not as, not explicitly, but it's a really good shout-out. So I think that the... I'm gonna weave that in in, in, like, two places. One is in the background agents, uh, week, and then also in the software factory week.”

Claim: The speaker plans to cover the SRE-related idea during both the background-agents week and the software-factory week.

“So absolutely right. Um, I do... It's a really good call-out, this idea of, like, trigger/event-based closing of the loop, um, which, you know, now is one of the, one of the probably two or three entry points when people talk about agents kicking off tasks in the software factory.”

Claim: Trigger- or event-based task initiation is probably one of two or three common entry points for agents to start tasks in a software factory.

“So, okay, I mean, like, I think the Resolve folks are doing well there. The Resolve is doing well. I would say Night Shift is also g- doing quite a, quite a good job there in terms of broadly, like, background agents.”

Claim: The speaker thinks Night Shift is doing a good job with background agents.

“So, okay, I mean, like, I think the Resolve folks are doing well there. The Resolve is doing well. I would say Night Shift is also g- doing quite a, quite a good job there in terms of broadly, like, background agents.”

Claim: The speaker thinks Resolve is doing well in the AI SRE area.

“When does it become... Like, where does, where does a- an SRE, an AI SRE, like, really see the most value? It's typically when you have, like, a tremendous number of, like, service dependencies and these, like, massive sort of, like, systems and tons of Kubernetes and Helm files, like, so, you know, uh, just, like, tremendously large Kubernetes infrastructure Helm that's, like, s- files kind of strewn ab- everywhere around.”

Claim: AI SRE systems tend to provide the most value in organizations with numerous service dependencies and very large, complex infrastructure, including extensive Kubernetes and Helm configurations.

“I think that's why, um, Resolve has done really well in, like, the enterprise, right? Where you go and you sell to Ford, you go and sell to, like, a Walmart, and you kind of say, "Hey, you have... Your service level dependencies are crazy."”

Claim: The speaker attributes Resolve's enterprise success in part to the complexity of service dependencies at large organizations.

“Um, and you see these, these, like, you know, these diagrams of how things are set up, and then they kind of go in and they're like, "Well, we'll, we'll try and make sense of all this." I think that, like, at, at smaller scales, it's usually still manageable, that you can kind of roll your own.”

Claim: At smaller scales, organizations can usually manage their systems themselves rather than relying on an external solution.

“But you know, I will say that for the enterprise, I think they, they've done a good job.”

Claim: The speaker thinks Resolve has done a good job serving enterprise customers.

“That's, that's right. That's, yeah, that's, that's the scariest thing. That's, that's absolutely right. It's, it's like, it's... And, and then you're, you're the person, you know, you're the person who's on call, and you're interacting with a system that goes down where n- that person still isn't there, and there's, like, old, out-of-date runbooks, and you're like, "What the hell is going on here?"”

Claim: At enterprise scale, on-call engineers may have to troubleshoot outages in systems whose original developers are gone and whose runbooks are outdated.

“It's a scary, scary thing. And yes, you're... At the, at the enterprise level, it's like, it's a whole, whole different beast.”

Claim: Enterprise-scale software operations are a substantially different and more daunting challenge than smaller-scale operations.

“Yeah. I, I mean, I think that it, it... Again, it's a, it's a function of how we just scale, right? And I think that for, for the enterprise, they, there are a number of issues that aren't always... I, I almost feel like the answer is not even always just more agents. Uh, and, and since we're still actually on, on the week five agent-ready code bases, like, when I talk to companies of, like, a certain scale, the, the things that are holding them back are really more just, like, low level, like, DevOps, CI/CD stuff, right?”

Claim: At companies of a certain scale, barriers to progress often lie in foundational DevOps and CI/CD issues, not simply in adding more agents.

“They're, they're like, their, their merge queues are getting backed up. They... It takes too long to- ... like, stand up an app.”

Claim: At some large companies, merge queues are backing up, and it takes too long to stand up an application.

“There's just, like, actually more foundational things that have almost nothing to do with... They're, like, throwing in one more SRE bot or a, a la, you know, a bot wouldn't be the answer to the problem because there's, like, more systemic things that they need to figure out.”

Claim: Adding another SRE bot would not solve problems caused by more foundational, systemic issues.

“Um, yeah, I think the, the, the biggest thing that maybe happened between last year and this year was just it suddenly became very clear that code review is a bottleneck, especially once you- people got really AI pilled and, and peop- there's the velocity on generation became so high that, you know, you could in theory, like, put up 30-plus PRs a day, uh, and then what?”

Claim: Between last year and this year, code review became clearly recognized as a bottleneck, especially as AI-assisted code generation accelerated.

“Um, yeah, I think the, the, the biggest thing that maybe happened between last year and this year was just it suddenly became very clear that code review is a bottleneck, especially once you- people got really AI pilled and, and peop- there's the velocity on generation became so high that, you know, you could in theory, like, put up 30-plus PRs a day, uh, and then what?”

Claim: With sufficiently high code-generation velocity, a developer could theoretically submit more than 30 pull requests per day, raising the challenge of reviewing them.

“Like, what's... Where, at what point do you decide that You either have to institute yourself and, and put yourself in the middle as some way to, to gate and review things in the traditional way, but then you're, you're literally the constraint, you're the choke point of the velocity, and that's not good, right?”

Claim: When a person must gate and review code in the traditional way, that reviewer can become the bottleneck limiting development velocity.

“So a lot of, I think, what I wanna touch on here is just that, that middle ground, right? Which is like there are, there is a class of problems that if you've put in thoughtful design into your agent readiness, you've put in thoughtful design around your verifiability, there's a class of problems that you can feel comfortable, you know, some percentage of the time essentially like auto-approving.”

Claim: For some classes of problems, thoughtful agent-readiness and verifiability design can support auto-approval in a portion of cases.

“I mean, like, like broadly right. Um, I guess you mentioned one thing like the last boss. Um, maybe not... I, I think there's two choke points. There's the review and there's probably- ... like the intent or like kind of the thing at the top, right? Like this is the-”

Claim: The speaker identifies code review and, probably, intent or the upstream specification as two bottlenecks.

“Um, but I think recently like the Amplitudes and, um, I think Cloudflare has, has done a good one. I think Augment has also done a good one where they talk about the design of their AI code review system.”

Claim: The speaker says that Amplitude, Cloudflare, and Augment have published posts discussing the design of their AI code-review systems.

“Um, these are a bunch of really good... If you, if you wanna share some of those links, I think they're fantastically written posts. Um, yeah, so shout out to... I actually, um, shout out to, to Cloudflare 'cause I think like, um, very like kind of like, you know, in a sense like legacy company or like, you know, bigger company, but I think they've done a fantastic job in terms of just their, their content around the AI, the AI SDLC.”

Claim: The speaker believes Cloudflare has done a fantastic job producing content about the AI software-development lifecycle, even though it is a larger, legacy company.

“Um, so I think they have a good post there. But TLDR, um, yes, I think this is true. I think that there's... We're still fi- finding that sweet spot. What is auto-approvable? What is not really auto-approvable? And if it is not auto-approvable, is even in that world of something that has to be somewhat human reviewed, what do we review? Are we still doing the line by line or are we doing some, some combination of the spec and... You know, I think this is... Yeah. Again, this is a whole separate question there as well, but-”

Claim: Teams are still determining which changes are suitable for automatic approval in AI code review, according to the speaker.

“What is not really auto-approvable? And if it is not auto-approvable, is even in that world of something that has to be somewhat human reviewed, what do we review? Are we still doing the line by line or are we doing some, some combination of the spec and... You know, I think this is... Yeah. Again, this is a whole separate question there as well, but-”

Claim: For changes that cannot be automatically approved, teams are still deciding what humans should review and whether review should be line by line or combine the specification with other information.

“Yeah. No. Well, I mean, okay, in a few different places, right? So like there's a spec that as it exists in like a linear issue, and there's a spec as it exists in like the context of a PR description, right? So you can like, you can have your peer description be the place where you like describe more or less the entirety of the implementation in a more condensed format. And as long as that's the most rep- that's the most up-to-date thing representing this point in time of the, of the code base, I think that's sufficient actually.”

Claim: A pull-request description can serve as a sufficient current specification if it concisely describes essentially the entire implementation and is the most up-to-date representation of the codebase at that time, the speaker believes.

“So we're thoughtful about like our peer descriptions. We make them be not just like, you know, a one paragraph. It's more like it is effectively like a spec of what has happened in the PR, and there's even like kind of line, um, file level pointers to things where things have changed, so you not only get the high level of what's happened, but even the kind of the, the construction of the files and sort of the surface level boundaries that you're touching.”

Claim: The speaker says the team writes pull-request descriptions as specifications of the changes, with both a high-level account and file-level pointers showing where changes occurred and which boundaries they touch.

“So yeah, I think that's where we, we've invested quite a bit.”

Claim: The speaker says their team has invested substantially in its pull-request descriptions.

“Have you, have you seen the, um, the post from Basis? Like they have a, they had a tweet, tweet little thing, a tweet article. I thought they did a great job talking about, um, like different sort of file artifacts. They, they, uh, they track via Git, and some of them are like the specs, the intent of what is to be, and the other is like what they call like the canonical state of the code, which is like a little bit more maybe what you're describing, which is implicit knowledge, implicit intent that also has to be codified somewhere, right?”

Claim: According to the speaker, Basis uses Git to track both specifications describing intended outcomes and a canonical code state that captures implicit knowledge or intent that must be codified.

“So that part, that's all version controlled, and I think they found that to work for them, so that might be another thing to share as well.”

Claim: The speaker says Basis has found its version-controlled approach to tracking specifications and canonical code state useful, and suggests sharing it as a possible approach.

“So I'll talk to, talk to him about that. I, I like that they're c- coming up with this. My solution for this is, um, I don't know if like how ready I am to talk about it 'cause I may never ship it, uh, is something that is replacing Git in order to-”

Claim: The speaker is considering a solution that would replace Git to capture specifications and related development information, but says he may never ship it.

“... capture all of this. Uh, so I call it a program database, uh, where it captures all traces and all... A- and, and it reifies specs to something that, that we, that we can ignore and, and keep, uh, and it also con- controls the skills that are attached to it, uh, as well.”

Claim: The speaker's proposed program database would capture traces, reify specifications, and control the skills associated with the system.

“Um, because I think like basically what Git... Git is only focused on code, and then, you know, you just need a source of truth that is, uh, not only, uh, everything else outside the code, including the, the implicit... Yeah, like implicit assumptions or like the constraints- ... the intent, the... Yeah”

Claim: The speaker argues that Git focuses only on code and that a source of truth should also capture non-code information, including implicit intent and assumptions.

“Uh, a- but also multi- uh, like highly concurrent. Uh, 'cause Git is, Git by intro- by design introduces mutexes everywhere- ... which sucks. Uh- ... so, so you need like super concurrency.”

Claim: The speaker believes a replacement for Git should support high concurrency because Git introduces mutexes throughout the workflow by design, which he considers undesirable.

“Scott, Scott. Um, so yeah. The, so the... I think there's a, there's a lot of people going after this. Um, and I think there's the, there's the question of, like, really what is the ideal representation of tasks? And I think the winner, probably not me, Linear.”

Claim: The speaker predicts that Linear will probably win in developing an ideal representation of tasks, rather than the speaker himself.

“Uh, and, like, clearly they have decided to become a, a, a coding agent company.”

Claim: The speaker says Linear has decided to become a company focused on coding agents.

“Yeah. Yeah. That, that is true. That is true. I, I wonder, I, I have, like, this somewhat, like, kind of held belief that I think a lot of companies are gonna evolve into some version of a coding agent company, which is any company that has, like, a, a interact with B2B software where you're collecting something about the user or some, like, product insight, I think you can also close the loop.”

Claim: Companies that interact with B2B software and collect user information or product insights can use those signals to take action and close the loop.

“Yeah. Yeah. That, that is true. That is true. I, I wonder, I, I have, like, this somewhat, like, kind of held belief that I think a lot of companies are gonna evolve into some version of a coding agent company, which is any company that has, like, a, a interact with B2B software where you're collecting something about the user or some, like, product insight, I think you can also close the loop.”

Claim: The speaker believes that many companies will evolve into some form of coding-agent company.

“'Cause ultimately, like, what a coding agent is, is the purpose of a coding agent is to act on some event or thing that's happening where a u- user's involved or a customer's involved, and they use that to, like, fix ideally some aspect of the product, right?”

Claim: A coding agent is intended to respond to events involving users or customers by ideally fixing an aspect of the product.

“I think what Linear is doing, I imagine the same thing, like, an Amplitude could do as well, right? Because they're also getting insights about user analytics, user problems. I imagine even, like, you wanna really stretch it, you could even have, like, a Decagon or CRM do something similar where, you know, "Here's what's broken. Go close the loop."”

Claim: The speaker thinks Linear, Amplitude, and potentially Decagon or CRM products could use user analytics or reports of problems to trigger actions that close the loop.

“I find that he's an incredibly thoughtful thinker, uh, thoughtful thinker as... You know, a thoughtful- ... thoughtful leader in sort of this space. I think Semgrep is an interesting company 'cause they've been around even before sort of agents, right?”

Claim: The speaker considers Isaac an exceptionally thoughtful thinker and leader in the relevant field.

“I mean, the, the OSS repo was w- w- existed well before that kind of s- traditional SaaS kind of, uh, defense mechanisms. Um, but I think they've done a good job of pivoting and, or not pivoting, or like augmenting their offering to, into this space.”

Claim: Semgrep's open-source repository predates the traditional SaaS security-defense mechanisms being discussed.

“I mean, the, the OSS repo was w- w- existed well before that kind of s- traditional SaaS kind of, uh, defense mechanisms. Um, but I think they've done a good job of pivoting and, or not pivoting, or like augmenting their offering to, into this space.”

Claim: The speaker believes Semgrep has successfully expanded or augmented its offering into the current security-and-agents space.

“Um, you know, I think it's, it's, um, it's an interesting time, right? Uh, there's just a lot going on where these, these attack vectors are now kinda coming to the surface. There's a lot more slop and AI-generated code, and I think maybe one of the core insights that I wanna drive here is just the idea of, like, dependency, kinda dependency tracking and supply management, basically.”

Claim: The speaker says that new attack vectors are becoming apparent as the amount of low-quality and AI-generated code increases.

“Um, you know, I think it's, it's, um, it's an interesting time, right? Uh, there's just a lot going on where these, these attack vectors are now kinda coming to the surface. There's a lot more slop and AI-generated code, and I think maybe one of the core insights that I wanna drive here is just the idea of, like, dependency, kinda dependency tracking and supply management, basically.”

Claim: The speaker identifies dependency tracking and software supply-chain management as a core insight for addressing security risks.

“Like supply chain, right? Um, you have to be careful about, like, your dependencies and, and pin things appropriately and don't just arbitrarily hit NPM install and expect the chain to go.”

Claim: Developers should manage dependencies carefully, pin versions appropriately, and avoid blindly running npm install and trusting the resulting dependency chain.

“That's absolutely right. That's absolutely right. A- a- and that's an interesting insight because it sort of to the limit, and, you know, I think it's, it's probably easier said than done, but if you believe that with code now being commoditized, code being easy, cheap, why not just build everything in-house, right?”

Claim: The speaker raises the possibility that, if code has become commoditized, easy, and cheap to produce, companies might build all their software in-house.

“Like that, like of course that's... And that would be the safest thing 'cause you've seen everything. You control everything instead of, you know, depending on Axios or whatever. But, you know, we'll see how many people actually act on that, but that is the safest thing.”

Claim: The speaker says that building all software in-house would be safest because the organization can inspect and control everything instead of relying on external dependencies such as Axios.

“Like that, like of course that's... And that would be the safest thing 'cause you've seen everything. You control everything instead of, you know, depending on Axios or whatever. But, you know, we'll see how many people actually act on that, but that is the safest thing.”

Claim: The speaker is uncertain how many organizations will actually choose to build all their software in-house.

“I, I always believe that, like, most OSS repos, if not all of them, will just become software factories that are effectively agent run in the future.”

Claim: The speaker expects most or possibly all open-source repositories to become software factories that are effectively agent-run in the future.

“What happens then where like, I mean, even, um, like, even if you l- look at Warp, right? Like Warp, which is a monocoding agent. Of course, they're also, you know, selling a software factory product, but, but they are running their Warp, open source Warp terminal as a software factory, and that almost suggests that there is a part where, yes, humans are still involved, but increasingly less so.”

Claim: The speaker says Warp operates its open-source terminal as a software factory and takes this as evidence that humans may remain involved in software production while becoming less involved over time.

“Certainly not every single person is reviewing every single thing that comes in to the maybe the same exact level that we did in the past. Does that make this code base more or less vulnerable to security?”

Claim: The speaker says that in the software-production process being discussed, not every person reviews every incoming change as thoroughly as people did in the past.

“It's a bit hard to say, right? Um, I think that's just where there's a little bit of nuance that I think we're still... And e- even the folks that I know that I s- that work in OSS are trying to figure out exactly what that stable point is.”

Claim: The speaker says it is unclear whether reduced human review makes a codebase more or less vulnerable, and that open-source practitioners are still trying to identify the right balance.

“... huge, huge, huge surface area thing. There's one argument from Malta of Vercel who's like, "Okay, guys, let's take every single, like, high depend, highly dependent on software, uh, that's written in C, C++ port it to Rust," and it's like, "We'll be faster, more secure." Uh, and, like, it, it won't... It'll probably take, like, $6 million, and but you'll j- you'll, you'll get it done. And, and then there's, there's another argument for... This, this also links to Pacing the Frontier from the, the open source, uh, from the Model Labs-”

Claim: One proposal is to port software that heavily depends on C or C++ to Rust, on the expectation that it will be faster and more secure, at an estimated cost of about $6 million.

“Uh, and, like, it, it won't... It'll probably take, like, $6 million, and but you'll j- you'll, you'll get it done. And, and then there's, there's another argument for... This, this also links to Pacing the Frontier from the, the open source, uh, from the Model Labs- ... who are like, "Okay, let's slow down so that we, uh..." Or like, "We'll do Daybreak. We'll do what's, uh, Glasswing. We'll make sure everyone's patched, and then we can let loose on rampant acceleration."”

Claim: Some model labs advocate slowing down until security initiatives such as Daybreak and Glasswing are completed and systems are patched, then resuming rapid acceleration.

“... there's, there's a, there's an inherent belief that the models will fix all security. And I think it's just unproven to me because what if they introduce new bugs, and they're just different bugs?”

Claim: The speaker considers the belief that AI models will fix all security problems unproven, because models might introduce new and different bugs.

“Yeah. Uh, I, I guess like that's almost like in the limit. It, there, there's this... The question is like, is, is a model... Do we believe a model will create perfect software, right? ... perfect software, then, then you're done. Then, then that's done, right? Then, then we have... Then we're done here at all. Like, then, then, then we're just gonna go home and, and, and chill on a beach for the rest of our lives.”

Claim: If a model could create perfect software, then software development would be finished, according to the speaker's hypothetical.

“You can give it for three months. Someone can try to exploit it for six months. The problem's not solved, right?”

Claim: Allowing three months for patching does not resolve the security problem if attackers can spend six months attempting exploitation.

“Yeah. And, and my whole problem with like... And, and again, like this is a whole rant that we don't need to go into- ... but like it's, like, you know, you name 100 plus organizations that like, okay, these are like priority. They, they have access to a s- to a cyber model that, uh, you've just created 100 attack vectors for me to go compromise. And like they're... By the way, they're pretty easy to compromise.”

Claim: Providing more than 100 organizations access to a cyber model creates potential attack vectors against those organizations, which the speaker says are fairly easy to compromise.

“Like, and, and like I think actually the security folks love it because they, like they usually don't get that much attention, and now they have like a ton of attention.”

Claim: Security professionals usually receive relatively little attention, but are receiving much more attention now.

“Obviously many, like, great players in, in, in that field, but it's, it's a tough, tough industry, I think. And so basically like, uh, the, the best time... But I think it's not controversial. Best time for any security company is when a, a high-profile incident just happened. Like then obviously you're gonna sell a bunch of security.”

Claim: A security company has its best sales opportunity immediately after a high-profile security incident, when customers are likely to buy more security products.

“Um, yeah. So, so I, I think, I think that's a, a, a fair thing. Like people, you, people should be aware. People should know the, the base- basic miti- mitigation, mitigations. Uh, I do think like s- code scanning is important.”

Claim: People should understand basic security mitigations, and code scanning is important.

“But like I, I just wanna give people the, the idea that like, oh, models introduce bugs that you wouldn't make as a human, and like if you're, you're not checking those bugs, then like-”

Claim: AI models can introduce bugs that a human programmer would not make, creating risk when those bugs are not checked.

“Forget bugs. Models are pretty good. They'll have, on average, I would say less bugs. The problem is also that you don't even know what you're putting out.”

Claim: The speaker estimates that AI models produce fewer bugs on average, but warns that users may not understand what code they are releasing.

“Yeah. And that's, yeah, I think we have to do top of mitigations and, and like the... And there's sort of a combination of security and, and security as part of code review as well, right? I mean, I think this is also like doing those checks I think is important.”

Claim: Security checks should be performed, including as part of code review.

“... is that, uh, not all parts of the code base are created equal. The parts of the code base where security is very important is data storage- ... um, and, uh, billing. So let's call it like data, billing, uh, auth, and auth is obviously related to all those things. And those are basically don't write code, and then everything else go, go nuts.”

Claim: Because security matters especially for data storage, billing, and authentication, the speaker recommends avoiding AI-generated code in those areas while allowing it more freely elsewhere in the codebase.

“A- and, and it's really funny how like... Okay, well, if you don't know what the difference between auth and an auth Z is, please learn it.”

Claim: People should learn the distinction between authentication and authorization if they do not already know it.

“And then, you know, if you, if you don't know... If you have never looked at AWS IAM, which you have, please learn it. Like this, like these are basic like conceptual things to understand that you can prompt for. Now, once you have the language, you can prompt for it. You don't have the language, you're fucked.”

Claim: People need to understand the language and basic concepts of technologies such as AWS IAM to prompt effectively about them.

“And then, you know, if you, if you don't know... If you have never looked at AWS IAM, which you have, please learn it. Like this, like these are basic like conceptual things to understand that you can prompt for. Now, once you have the language, you can prompt for it. You don't have the language, you're fucked.”

Claim: Learning AWS IAM and related security concepts gives people the vocabulary to prompt AI systems for those concepts; without that vocabulary, they cannot prompt for them effectively.

“So DNS is more of like a reliability uptime.”

Claim: DNS is primarily a reliability and uptime concern.

“Yeah. That I would, yeah, probably put that above that for sure. Only DNS 'cause of uptime. Yeah, general uptime.”

Claim: The speaker considers security incidents involving financial loss or customer-information leaks more important than DNS issues, which primarily affect uptime.

“But yeah, week eight, um, Rajesh, who has been leading a lot of like the infra stuff, uh, at, uh, kind of cloud infra, uh, DevX it's, and, and beyond at Cloudflare, which as I earlier said, was I think they're doing a great job here, and definitely their adoption for a company of their size and a public company of their size is, is fantastic.”

Claim: Cloudflare's adoption of these practices is excellent given its size and status as a public company.

“And then their content I think is also really good. I mean, this is fundamentally about all the ways you can use like asynchronous background agents. Um, I think I'm also gonna spend quite a bit of time talking about, you know, the Stripe Minions, the Spotify, I forget the name of their version of this, um, the Ramp Inspects, the, you know, things of that nature that I think are all kinda showing what does it look like when you scale this out.”

Claim: The speaker plans to discuss asynchronous background agents and examples including Stripe Minions and Ramp Inspects to illustrate how such agents can be scaled.

“Yeah. I mean, I guess less on consumer, more on like, you know, software teams using them at scale.”

Claim: The speaker's discussion of these agents focuses less on consumer applications and more on software teams using them at scale.

“Yeah. It's like Grokbot is one of those interesting things where like it very, very much presented as consumer-y Uh, it's, it's Groq under-- It's, you know, it's, uh, it's Cursor under the hood.”

Claim: Grokbot is presented as a consumer-oriented product while using Cursor under the hood.

“They do.”

Claim: Cursor offers a code-based integration.

“Uh, which, uh, I think that's the point of what they're trying to do. They're, they're trying to be like, uh, well, you can use it for consumer stuff, but here's the extra power if you know how to use it well, uh, because, because it's actually Cursor under the hood.”

Claim: Grokbot aims to support consumer tasks while giving skilled users access to greater capabilities through its Cursor foundation.

“Yeah. So, so a- and I'll give you another sort of practical example. Uh, so I, I run all my SEO and AEO inside of Codex, and it has a code base- ... even though like a traditional marketer would just check Ahrefs or something.”

Claim: The speaker performs all of his SEO and AEO work in Codex using a codebase.

“Um, because I write all these scripts for doing all the-- all sorts of crazy stuff.”

Claim: The speaker writes scripts in Codex to automate a wide range of tasks.

“And, and also I need to store data, and also I need to visu-visualize data. And like pretty soon you go f- like, as long as you're doing anything on a ongoing basis that has like maybe a lot of steps or a lot of data, you need a code base.”

Claim: Ongoing work with many steps or substantial data requires a codebase for tasks such as storing and visualizing data.

“... that is using Claude Code or just Claude in the app now is working in a code base that they would never know, never touch, but there's a code base management.”

Claim: A nontechnical marketing user may use Claude Code or Claude in the app while an underlying codebase is managed without the user knowing about or interacting with it.

“I think Claude Tag is a good one for, uh, background agents.”

Claim: Claude Tag is a good example of background agents, in the speaker's view.

“... established-- Uh, I'm bullish on MCP. Uh, it, it's-- There's not bullish or bearish, it just is. It just is a part of the landscape now-”

Claim: MCP is an established part of the current technology landscape.

“... unproven business. Uh, I st- I thought about starting one. Uh, I actually pitched the guide Anthropic on, on potentially starting one- ... and then I lost interest. I lose interest in a lot of, a lot of things. But like, uh, what, what's, what's in there?”

Claim: The speaker considered starting an MCP portal business and pitched the idea to Anthropic, but later lost interest.

“And so here, uh, I think that something like MCP portal really becomes interesting once you're at a certain scale where you effectively have to have a single point of entry to do things like governance, security, auditing, cost optimization.”

Claim: MCP portals become useful at sufficient organizational scale, when a single entry point is needed for governance, security, auditing, and cost optimization.

“Uh, and so, you know, I, I'd be curious to hear what you see as the future or maybe lack thereof of something like this. But it to me is-- And it's if you believe in like an open router or, you know, some place-- like a single point of entry for, for many things to happen and connect to and be effectively, you know, governed, I feel like there's a world where this also can easily exist.”

Claim: If an open router or similar single entry point is used to connect and govern many systems, an MCP portal could plausibly exist as part of that architecture.

“Yeah. My, my only tweak on this is that this folds into the LLM gateway. So they, they- ... they are the exact same platform.”

Claim: MCP portals and LLM gateways belong to the same platform, with MCP portal functionality folding into the LLM gateway.

“Uh, and, and then, then by the way, it also folds into the other thing of, um, what did I say it was? It was, uh, off Z, you know- ... permissioning and all these things- ... and billing. So like e- like all-- To the limit, all infra converges to the same exact problems where they have the box in the middle, and then you have the box in two sides. Uh, the, you know, God box will solve everything, right?”

Claim: Infrastructure systems converge on a shared intermediary layer that handles functions such as permissioning and billing between connected systems.

“And it's true only if you take-- uh, if you understand the job of the God box. And so separating MCP portals and LLM gateways means you don't fully accept that they are, uh, you know, the, the, the same layer problem.”

Claim: Treating MCP portals and LLM gateways as separate systems reflects a failure to recognize that they address the same layer-level problem.

“Uh, in, in a very similar way, I also find that people don't understand database querying, uh, to have a lot of the similar things where they will separate filtering from ranking- ... from, uh, uh, or, or just like general recommendation. They're the same problem.”

Claim: Database filtering, ranking, and general recommendation are aspects of the same underlying problem.

“Uh, and, and so like then you just combi-- Uh, because like when you take all these things, you combine them in the same layer, then you can sort of cross-depend on, on things in useful ways. If they're different layers, then they cannot.”

Claim: Placing related functions in one layer enables useful cross-dependencies that are unavailable when those functions are split across layers.

“... architecture. Uh, so anyone working infra needs to understand, uh, congratulations, you now have a job in infrastructure, or you want to be an infra engineer. You need to understand that your entire job is making God boxes and then taking them apart.”

Claim: Infrastructure engineers need to understand their work as building integrated systems (“God boxes”) and later decomposing them.

“So either, yeah, your problem is either you don't have enough... You don't have a good gut box, or your problem is someone else's gut box you don't like, so now you have to take it apart and make your own.”

Claim: Infrastructure work often involves either creating a suitable integrated system or replacing and rebuilding an existing system that does not meet one's needs.

“Which, which is a... This is a, a, a restatement of, like, there's only two ways to make software, which is- ... you bundle them, you unbundle them.”

Claim: Software development can be understood as bundling components together or unbundling them.

“But, like, what are you bundling? It is the things that go together, hopefully.”

Claim: A sound bundling decision groups components that belong together.

“And then, like, maybe o-over time, people's opinions shift as what, what goes together, what should not go together.”

Claim: People's views about which components should be bundled together or kept separate may change over time.

“So, so, like, there, you know, there, there is that, that whole thing of, you know, maybe, uh, Monaco should be a more of a point solution, maybe Monaco should own everything, right?”

Claim: The speaker considers two possible product strategies for Monaco: focusing on a point solution or owning the entire product space.

“And, like, there's the question of, like, in some ways, you're bundling for, uh, the, the kind of startup segment that you serve. In some ways, you're unbundling from Salesforce because Salesforce that... And then that's how you win market share from Salesforce. You unbundle from Salesforce.”

Claim: Monaco's strategy involves bundling products for its startup customer segment while unbundling offerings from Salesforce.

“And then that's how you win market share from Salesforce. You unbundle from Salesforce.”

Claim: Monaco can gain market share from Salesforce by unbundling Salesforce's offerings.

“Never thought it, never thought, thought about it that way. But yes, uh, and you know, one, one of those things, so, so we just shipped the OpenRouter pod today, right? And, um, OpenRouter, huge success.”

Claim: OpenRouter has been highly successful.

“Lots of people tried to build gateways. None of them as successful as OpenRouter. OpenRouter only builds gateways. So there's an example of, like, this... They disagree with me that the, the o- the router layer should also do ops-”

Claim: The speaker says OpenRouter was more successful than other efforts to build gateways.

“Lots of people tried to build gateways. None of them as successful as OpenRouter. OpenRouter only builds gateways. So there's an example of, like, this... They disagree with me that the, the o- the router layer should also do ops- Uh, and they, they just, they only handle model routing. And so, like, I, I, like I am... I love being wrong like that because they're like, "We are the point solution, and we only do this one thing really, really well."”

Claim: OpenRouter focuses exclusively on gateways and model routing.

“... because Arise is doing ops, um, and observability and, and what have you. And, and they... And like the MCP gateway, they don't have that.”

Claim: Arise provides operations and observability capabilities that the MCP gateway lacks.

“Uh, and they, they just, they only handle model routing. And so, like, I, I, like I am... I love being wrong like that because they're like, "We are the point solution, and we only do this one thing really, really well."”

Claim: The speaker welcomes being proven wrong when a company succeeds by focusing on one point solution and doing it very well.

“And, uh, that's our, that's our en-engineering focus, which is good, but it's also the marketing focus, which is nice.”

Claim: The speaker says the company's narrow engineering focus also serves as its marketing focus.

“Mm-hmm. I, I think the, um... This becomes... I think the, the what you've described of like the one, one-stop shop for all the things is ideologically what makes sense.”

Claim: The speaker thinks consolidating all relevant tools into a one-stop shop makes sense in principle.

“But I think what ends up happening in practice, and you know, part of it is probably also marketing, is that especially when you look at the enterprise, is like this is broadly under infra, like broadly under, under platform, but then there's separate teams owning different parts of the stack.”

Claim: In enterprises, separate teams often own different parts of the stack despite those parts broadly belonging to infrastructure or platform.

“And I think part of that consolidation at the, at the scale of an enterprise, if we're going to hit MCP portal and the gateway, someone has to come together and say like, "Now we're... There's a single endpoint that you're hitting- ... for both of these things." And that fundamentally is like a hard thing to, you know, get- ... everyone on board with.”

Claim: At enterprise scale, consolidating an MCP portal and gateway requires coordination to create one endpoint for both.

“... for both of these things." And that fundamentally is like a hard thing to, you know, get- ... everyone on board with.”

Claim: The speaker says it is difficult to get everyone on board with consolidating the MCP portal and gateway behind one endpoint.

“Um, so oftentimes it's like more like political and, and even like people problem rather than like a, from a software, pure software design perspective, this makes a ton of sense.”

Claim: The speaker says organizational politics and people problems often obstruct consolidation more than software-design considerations do.

“But that's not it, right? They, like software systems sometimes are bottlenecked by people problems more than, than pure software.”

Claim: The speaker says people problems can be a greater bottleneck for software systems than technical software-design issues.

“Yeah. The way, uh, my, my framing of this is typically, you know how you ship your org chart? This is, uh, well, I forget what the, the name of this principle is. Uh, you know, Conway's Law, you ship your org chart.”

Claim: The speaker invokes Conway's Law to express that a system's design reflects the organization that builds it.

“Uh, you buy your org chart. Like, like, like who... Like when you start a startup or you st- or you make a tool, who in the company is your, are you best friends for?”

Claim: The speaker says a startup or tool builder should identify which people inside a company they are closest to.

“And if it's for everyone, it's for no one.”

Claim: The speaker asserts that a product aimed at everyone effectively serves no one.

“... a, a conscious part of, uh, latent space in A- in AIE is like, okay, there's this category of software engineer that really, really, really cares a lot about AI.”

Claim: The speaker identifies a category of software engineers who care deeply about AI.

“Is it gonna make it their day job? Let's only make it for them.”

Claim: The speaker proposes targeting the product specifically at software engineers who care deeply about AI, conditional on their making AI their day job.

“Yeah. I, I... Look, we talk about databases. Talk about security, but like there's an A- there's an AI angle to it.”

Claim: The speaker says topics such as databases and security can be approached through an AI-related angle.

“... one thing I'll just like shout out as well related to the, to the point is, uh, I think role of the future that is increasingly important, um, platform engineer.”

Claim: The speaker expects the platform-engineer role to become increasingly important as a future role.

“If I hadn't started AI, I would've done platform engineer.”

Claim: The speaker says that if they had not started working in AI, they would have pursued platform engineering.

“Uh, because that is the role that deci- that decides all of this mission-critical stuff, and then the kids over in applications can do whatever they want on top of our platform. But, but, you know, the big boys do platform.”

Claim: The speaker says platform engineers make decisions about mission-critical systems, while application developers build on the platform.

“Yeah, and I think in the modern day, they own, they own cloud, they own DevEx- ... they own security, SecOps. So I think it's, it's all the things.”

Claim: The speaker says modern platform teams own cloud infrastructure, developer experience, security, and security operations.

“Like everything where you can actually like lose the company or like, or make or break, like yeah, platform guys. And then product guys can do, you know, adjust the CSS. Uh, they're not gonna-”

Claim: The speaker says platform teams are responsible for areas where failures can threaten a company or determine its success.

“Uh, I want people to, to really start thinking about this, this world of, of this machine that builds the machine and where, if at all, are people still in the loop there.”

Claim: The speaker wants people to examine software factories as systems that build software systems and ask where humans remain involved.

“You know, today, I think the answer is some a- some combination of intent and kind of review or maybe the places like... So intent where you're defining the specs and what needs to be built and, and review where you're, you know, s- doing some amount of your own checking of the system.”

Claim: The speaker says that, today, humans remain involved in automated software systems through some combination of defining specifications and reviewing the system's output.

“And of course, this also... This is, I think, that bottleneck is gonna be loosened up before even the intent piece does. But for now, those are the two choke points, I think, in, in a fully automated system.”

Claim: The speaker expects the review bottleneck in fully automated systems to ease before the intent bottleneck; currently, intent and review are the two main choke points.

“Um, so yeah, I just wanna give people exposure to that. For one of the OSS partners, actually, uh, Warp, they're doing this thing as part of their contribution to the OSS program where they're gonna essentially make a version of Warp that is running the factory, and students will just be like building the factory, contributing to the factory- ... and keep it running.”

Claim: As part of its open-source program contribution, Warp plans to create a version of Warp that runs the software factory.

“Um, so yeah, I just wanna give people exposure to that. For one of the OSS partners, actually, uh, Warp, they're doing this thing as part of their contribution to the OSS program where they're gonna essentially make a version of Warp that is running the factory, and students will just be like building the factory, contributing to the factory- ... and keep it running.”

Claim: Students participating in the program will build, contribute to, and maintain the software factory.

“And so that's gonna be one of the ways. And then they're gonna, yeah, find things that they find that are interesting there and pull it back in.”

Claim: The speaker says participants will find interesting ideas or work in the factory and bring them back into their own work.

“Yeah. I, I think that's, that is... It, it is a good question, and I think they're, you know, of course, entitled to kind of r- try and sell the products they do. I think it's a hard system to own its entirety. It's a, it's a lot. It's, it's a lot, right?”

Claim: The speaker says owning an entire software-factory system is difficult because of its scope and complexity.

“Yeah. It is a lot. And, and I'm, I'm trying to... You know, I haven't, like, gone too deep in understanding, like, the, the, how they approach their GTM here, but I suspect you would have to do it piecemeal, right? You would have to be like, "Hey, we first do a coding agent." Oh, then also... Like, you basically land and expand, right? I mean, we talked about this bundling, unbundling piece. You don't start by saying, "Here's your whole factory."”

Claim: The speaker suspects that enterprise AI software sales would need to proceed piecemeal, landing with an initial product and expanding over time.

“There's no way you could get, like, a Ford on board with that. You know what I mean? Where you're gonna go on and just plop the whole thing on their, on their desk. Um, that's at least how I would think about it, is you first be their, their coding agent of choice.”

Claim: The speaker believes an enterprise software factory would be difficult to sell to a company like Ford as a complete package, and suggests starting with a coding agent instead.

“You, you, you build out some backroad agent for them. You, you own some of their SRE pieces, and then you're like, "Well, now we can just weave it all together if you define everything in code," or something like that. Like, you know, configs.”

Claim: The speaker proposes expanding an enterprise AI offering from a background agent and SRE work toward an integrated system, enabled by defining components such as configurations in code.

“Yeah. Absolutely. Yeah. I mean, some version of this is gonna exist. Uh, just to fix this point, I do wonder about the, the sales piece here, like, the actual business of this.”

Claim: The speaker expects some version of an AI software factory to exist, while expressing uncertainty about how its business and sales model will work.

“That's an interesting question. But from a principles perspective, I think this is the, the, this has been version of the future, and then beyond that, I think there's gonna be some amount of dabbling into and speaking about, like, okay, what does it mean to... Yeah, what does it mean to be a developer in, in the future, right? Like, are we... People talk about taste. People talk about, like, you know, understanding of systems and architecture and, and yeah, like, software taste.”

Claim: The speaker views the software-factory model as one possible future and considers how the developer role may evolve, including the importance of software taste and systems and architecture understanding.

“Yeah, so I mean, the factory is, is, like, the tactical, but I think beyond that, there's, like, a, there's a question around, like, the role, right? Like, what does the role evolve into? Um, if, if the factory is, is running and, you know, full steam ahead doing its thing, the conveyor belt is, is churning out code, what, what do we do in all this, right?”

Claim: The speaker raises the question of what human workers will do if an AI software factory operates at full speed and continuously produces code.

“... around the factory. But yeah, I, I, I think that there was this part of-- At least the discussion points there, which I am pretty much on, on board with, is, like, increasing levels of abstraction and then somehow there's someone still, even a CEO of a company has to have enough oversight and understanding of, like, different constituent parts to be able to make a judgment on direction, right?”

Claim: The speaker agrees that increasing levels of abstraction still require an overseer, potentially even a CEO, to understand the system's constituent parts well enough to judge its direction.

“To be able to course correct, to be able to set metrics, and things of that nature. I think there's a version that translates of that. There's a, there's version that, uh, of that analogy that translates to, like, someone who oversees the technical.”

Claim: The speaker says that an overseer of an increasingly abstract technical system needs the ability to course-correct and set metrics.

“We call it a manager of agents. We call it a, you know, different terms. They're all buzz, buzzword-y, but, um, in some, some respects, someone who understands product requirements, constraints, technical constraints, kind of loops through all of those and, and, and knows how to adjudicate.”

Claim: The speaker describes a possible agent-management role whose responsibilities include understanding product requirements and technical constraints and adjudicating among them.

“The, the thing that I care, uh, I, I'm thinking a lot about is... Well, so one part of this discussion that resonates more with me is the loop, uh, stuff. Where like, um, okay, you, you started writing prompts. Now you no longer write prompts. You write prompts that write prompts. And then you, you just keep going up. And I, I think, to me, the sort of recursive, concentric circles nature, going outward but also going inward- ... is something that builds the factory. I think you build up to the factory-”

Claim: The speaker sees recursive layers of prompt-writing—where prompts generate other prompts—as a process that builds toward an AI software factory.

“But I think a lot of people will eventually start building down. Like, you start from the top level, like, "What is my goal?" And then you figure out the systems. Um, that currently creates a ton of slop, and you should not do that. But someday you will.”

Claim: The speaker predicts that people will eventually build systems downward from a high-level goal, but says this approach currently produces substantial slop and should not be used yet.

“What is the sort of prime question that people exist to solve? Like, that is your, that is your thing, right? So i-in, let's, let's say an engineer today, um, you are measured by your PR merge rate. PR contribution rate. Well, two years from now, that's gonna be ridiculous. Okay, well, what's your main thing?”

Claim: The speaker predicts that within two years, an engineer's pull-request contribution rate will be an absurd measure of their work.

“People always go to money too quickly. It, it's something in between money and- ... PRs.”

Claim: The speaker argues that money is too simplistic as an answer to what engineers should be measured by, and suggests the relevant measure lies between money and pull-request output.

“But, but if the, but if the engineering is evolving into, like, more of a PM-engineer, like product engineer, you know, hybrid, then they... Then I think that's, you know, that's my answer to the question of, like, what does the role evolve into.”

Claim: Engineering may evolve toward a hybrid role that combines product management and engineering.

“I think it goes without say that, um, I would today recommend to students spend more time thinking about holistic systems, not just your, like, single part of the, the engineering puzzle, but how that ties in with...”

Claim: The speaker recommends that students think about holistic systems and how individual engineering components fit together, rather than focusing only on one component.

“And, you know, and this is, like, like, in practice, ends up happening. Like, students for their capstone projects, I encourage them to go out and, like... In fact, what I give them as the seed of ideas for final projects is, like, the, the YC request for startups.”

Claim: The speaker encourages students to pursue their own capstone project ideas and gives them Y Combinator's Request for Startups as a source of final-project prompts.

“I'm almost like, "Go build a company." Like, you know what I mean? Like, "Go, go build something interesting" that-”

Claim: The speaker encourages students to build interesting projects with an entrepreneurial, company-building orientation.

“Uh, I mean, a variety of things. Like, someone was like, um, uh, someone came in the E department, uh, uh, was in the E school, uh, and E-- was studying EE, and they were like, they built effectively a translator for, like, parsing of, uh, like, manuals to... that have, like, um, like, memory addresses in, in C, and then converting that into, like, uh, like, code that they could then, like, interpret and even, like, visualize.”

Claim: A student built a tool to parse manuals containing memory addresses in C and convert their contents into code for interpretation and visualization.

“So, so something that was specific to their, their use case. Someone was, like, a designer and is now actually at Vercel, and she, like, had, you know, like, take a picture of my, like, design notebook and generate me an app based on what I, I take a picture of, right?”

Claim: A student designer built a tool that generated an app from a photograph of her design notebook.

“So literally, she was like, for her demo, like, "Here you go." ... snaps a picture of, like, some random, like, scribbles that she was doing, and she got a fully fleshed out Vercel app. And from that, and I think because of that, probably got a job at Vercel doing that, doing, doing some version of this.”

Claim: In a student's demo, a photograph of handwritten scribbles was turned into a fully fleshed-out Vercel app.

“So literally, she was like, for her demo, like, "Here you go." ... snaps a picture of, like, some random, like, scribbles that she was doing, and she got a fully fleshed out Vercel app. And from that, and I think because of that, probably got a job at Vercel doing that, doing, doing some version of this.”

Claim: The speaker thinks the student probably obtained a job at Vercel because of her project, working on a related version of it.

“Yeah, yeah. And I think she's now working on Eve, uh, from what I remember.”

Claim: The speaker recalls that the student is now working on Eve, though he qualifies this recollection.

“Eve? Eve is an OSS, uh, or, like, OSS partner. I mean, like, like, they-- One of the, the ways Vercel is partnering with us is that they are having- ... students build with Eve. Yeah.”

Claim: Eve is described as an open-source partner; Vercel partners with the speaker's organization by having students build with Eve.

“I think at some level, students should also stay grounded and, like, there's a lot of stuff happening at the edge. There's also a lot of companies that are still, like, single call ChatGPT in production, playing around, haven't tried it.”

Claim: Many companies remain at the stage of using a single ChatGPT call in production or experimenting without having tried more advanced approaches.

“Um, you know, there, there is a lot of that, so there's, there's still time for it to work. But I think your point on taste is good. Have taste, try stuff.”

Claim: The speaker believes there is still time for companies to make progress in adopting AI tools and approaches.

“Um, you know, there, there is a lot of that, so there's, there's still time for it to work. But I think your point on taste is good. Have taste, try stuff.”

Claim: The speaker advises people working with AI tools to develop taste and experiment with them.

“Like, uh, whether each company has their own version of a software factory, uh, you're building tools to help companies do it, like, staying on top of it, actually using the tools, playing with it.”

Claim: The speaker describes possible paths in which companies develop their own software factories or use tools built to help them do so, and recommends staying current and actively using the tools.

“Like, you'll, you'll figure it out along the way, you know?”

Claim: The speaker expects people to work out how to proceed as they gain experience with the tools.

“Yeah. And I think the, the... Uh, 100% agree with all that. I, I mean, first to your point that are a lot of companies still in the prompt ChatGPT or some version of, you know, ChatGPT, maybe they just got a cursor license and are just now getting people onboarded.”

Claim: Many companies are still using ChatGPT-like systems through prompts, while some have only recently acquired Cursor licenses and started onboarding employees.

“Um, that is way more true than we believe, certainly here in the Valley. Um, that is- ... I can, I can say with 100% confidence. Um, but yes, I think for the students being exposed to this is just, at a certain point, this, the fact that they know this and are on the edge is what will eventually allow them to also be some of the more interesting and more valuable contributors when they go to maybe places where, that maybe aren't up to snuff in terms of being on the frontier.”

Claim: The speaker says it is much more common than people in Silicon Valley believe for companies to remain at an early stage of AI-tool adoption.

“Um, that is way more true than we believe, certainly here in the Valley. Um, that is- ... I can, I can say with 100% confidence. Um, but yes, I think for the students being exposed to this is just, at a certain point, this, the fact that they know this and are on the edge is what will eventually allow them to also be some of the more interesting and more valuable contributors when they go to maybe places where, that maybe aren't up to snuff in terms of being on the frontier.”

Claim: The speaker believes that students exposed to frontier AI developments may become more interesting and valuable contributors at organizations that are not yet at the frontier.

“Yeah. I mean, so, so, I, I, but I do th- so Steve is n-not necessarily someone to take literally, but directionally, like, you know, directionally correct.”

Claim: The speaker considers Steve Yegge's ideas directionally correct, while cautioning that Yegge should not necessarily be taken literally.

“And, and so I think he's directionally correct on this one. And so if, if you're at the ID stage, it's time to move into the power user stage.”

Claim: People at the ID stage should move to the power-user stage.

“If you're a single agent, you probably, time to move to multi-agent, and blah, blah, blah. Uh, and you probably should try to build your own orchestrator, right?”

Claim: A person using a single AI agent probably should move to a multi-agent setup.

“If you're a single agent, you probably, time to move to multi-agent, and blah, blah, blah. Uh, and you probably should try to build your own orchestrator, right?”

Claim: People using a single AI agent probably should try building their own orchestrator.

“As an employer, I have this problem in my company today, which is I do not want to hire anyone who hasn't been through psychosis and come out the other end. Because I am currently paying for somebody to go through psychosis.”

Claim: As an employer, the speaker says he does not want to hire people who have not experienced psychosis and recovered, because he is currently paying an employee who is going through psychosis.

“And it's very expensive- ... from his salary, uh, very expensive in tokens, and I'm getting nothing out of it, and he doesn't get it. Like, my, my feedback doesn't get through.”

Claim: The speaker says an employee's psychosis is costly in salary and tokens, produces no benefit for him, and the employee does not take in his feedback.

“And so, like, uh, it's almost like I would encourage your students to have psychosis and then come out and go like, "Okay, what was wrong with me?" Right? 'Cause, like, that's what S-Steve just went through, like, a very public two-year journey of this.”

Claim: The speaker says he would almost encourage students to experience psychosis and afterward reflect on what was wrong with them, citing Steve's very public two-year journey as an example.

“It's weird on the student side. There's, like, a lot of, uh, industry. You think you're being productive, you're not, is probably the student version, right?”

Claim: Students may think they are being productive when they are not; the speaker describes this as the student-side version of the problem.

“Your notion charting and making all these optimizations, personal ads, and it's like, "Dang, I went through the psychosis, but nothing has changed," right?”

Claim: Students may make Notion charts and personal optimizations but still feel that nothing has changed after their period of psychosis.

“Right. Yeah. Uh, I haven't seen this. I've seen a version of this from Boris, actually. Obviously, more geared toward cloud use cases, but I remember he made, like, a cloud artifact which showed, uh, I think more at the team level. But I think he had four stages, not eight stages, but it kind of rough-roughly followed the same progression of, like, no adoption to through to, you know, I guess full adoption.”

Claim: Boris's artifact has four stages rather than eight and, in the speaker's view, roughly follows a progression from no adoption to full adoption.

“Yeah. Only four is good. This is a, this is too many levels. Yeah. Well, see, we'll see if, uh, if we can pull it up. But other than that, you know, zooming out, have we, you know, discuss-discussed all the things-”

Claim: The speaker considers four stages preferable and says the artifact being discussed has too many levels.

“I, I would give a shout-out to dynamic workflows. This is one of those things that is a sub-agent pattern- ... that the OpenAI guys are ex- also exploring, um, that is clearly fitting And, uh, and so basically you're writing a harness on demand for the thing that you're trying to do.”

Claim: Dynamic workflows are a sub-agent pattern in which a harness is written on demand for the task at hand.

“And so, uh, this is maybe-- like a lot of sub-agent conversation is just you're copying the, the parent agent- ... but you actually should be writing way more code and then, like, running that as a, as a, as a sub-agent, and that is what the Anthropic people have chosen to call dynamic workflows.”

Claim: Instead of simply copying the parent agent's conversation, the approach is to write more code and run it as a sub-agent; Anthropic calls this a dynamic workflow.

“I will note that when Boris wrote this, it's for team adoption, for people that are listening, and also he says, you know, "Anthropic is on stage three. We're still trying to get to stage four."”

Claim: Boris's team-adoption artifact places Anthropic at stage three, with Anthropic still trying to reach stage four.

“He's like, "I am at stage four. I'm the guy." But you know, it's, it's not as simple as Steve's. Like Steve is a very good personal do this, do this, do this. Uh, this is a much more if you're running a team and you wanna AI pill your team and not just- ... be a user of one.”

Claim: Boris's artifact focuses on adopting AI across a team rather than on individual AI use.

“No, this is, uh... Thanks for-- I mean, ton of fun. It's, it's been a, a, a, you know, nine months in the making. I'm excited. This is, uh, end of week one. A lot to do, a lot to learn. I'm excited to see what happens with the-- especially the OSS partners and just getting students, like, deep in, in the code of, of some complex code bases, um, having them build some stuff. So yeah, it'll be fun.”

Claim: The speaker is excited to see the outcomes of open-source partnerships and students working deeply in complex codebases to build projects.

“Um, yeah, I, I work at AI credit card. Yeah. No, I, I, I think. Look, I will start like, uh... I remember, you know, it was like, you know, there's a certain number of credits for sure, but then on top of that, he's like, "Look, if they wanna go hog wild with this, like, let me know and we'll do-- have these, like, one-off experiments where they can just, like, scale up to, like, a thousand plus agents." And so-”

Claim: Students have a baseline allocation of credits, and a partner offered one-off experiments that could let them scale to over 1,000 agents if they wanted to use resources extensively.

“Um, those people who are ambitious and, like, can actually execute- ... those are very valuable, and that's basically a-”

Claim: People who are ambitious and able to execute are very valuable.

“Yeah, I, I, but I, I do think, like, students should, should, like, understand how to be ambitious because snapping a photo-- I mean, respectfully, snapping a photo and turning it into an app is not that ambitious in, in th-this day and age.”

Claim: Students should learn to pursue ambitious projects; in the current era, turning a photo into an app is not very ambitious.

“Uh, writing a browser is.”

Claim: Writing a browser is an ambitious project.

“And there-- and, like, there actually is a lot more projects like that.”

Claim: There are many other projects comparable to writing a browser.

“Yeah. Yeah, I think that's right. Yeah. It's all about raising the floor. I mean, I think, yeah, last year it was, uh, it would... You know. You know, credit's credit to the student of, of last year, but, uh, yeah, this year I obviously we're, we're gonna expect- We're gonna expect more. I mean, yeah. But that's, but that's, that's the whole point, right? Like we're leveling up, right? Last year it was like, "Yeah. Hey, have you heard of prompting? Like, here's what it is." I mean, I spent some time talking about like, you know, K-shot prompting as one of the lessons, and like I don't even think you guys even think about that now, so, um-”

Claim: The course is raising expectations: students are expected to do more this year than they did last year.

“... there's a version of thinking you stick out in your cohort of classes, and then there's also a version of you're also in early engineering roles with a bunch of people in industry, people still trying to get jobs, and it's not, it's not a, you know, not to call out one demo, but you can, you can do a lot to stick out.”

Claim: Students can stand out both within their class cohort and among people in industry or seeking engineering jobs.

“I'll, I'll give, uh, students a freebie, uh, of a week 11. Uh, make benchmarks. Make benchmarks and evals. Uh, this will get you, uh, a, a good one, are gonna get you in the door of every frontier lab and every front, every top agent lab.”

Claim: Building benchmarks and evaluations is a strong student project that can help students get in the door at frontier AI labs and leading agent labs.

“Yeah. Um, and probably also there's like big... a lot of money to be made in that as well. So not just, uh, in the doors, but if you make that into businesses, you, you can, uh, you can really, really cash in on that.”

Claim: The speaker thinks there is probably substantial money to be made by turning this kind of work into businesses.

“Yeah. People don't know, like C- C-Bench was a undergrad project.”

Claim: C-Bench originated as an undergraduate project.