Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

“The easiest way to put it is we build AI infrastructure. So we, we offer a bunch of compute, a, a compute platform and a storage platform that enables a lot of customers like Decagon and Suno and Runway and Co-Cognition and many others to, to essentially build AI applications.”

Claim: Modal provides AI infrastructure, including compute and storage platforms, for customers building AI applications.

“So what that means in practice is a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, uh, sometimes batch jobs and, and other things.”

Claim: Modal's platform supports AI inference, sandbox workloads, training and fine-tuning, and batch jobs.

“So we have basically built a, um, a software SDK that enables customers to move really fast and, and build this.”

Claim: Modal's software SDK is designed to help customers build applications quickly.

“Uh, we manage all the capacity underneath, make it easy to deploy and, and scale and, and run these things in production. So you were on Latent space [on-hold music] like two years ago. A lot has happened since then.”

Claim: Modal manages underlying compute capacity and provides tools for deploying, scaling, and running workloads in production.

“But it's like, it's probably like at least like fifty x- Yeah -in terms of like revenue and users and usage- Yeah, yeah -and stuff like that. Yeah. I use it actually for like a side project that I have where, uh- Nice.”

Claim: Modal has probably grown by at least 50 times in revenue, users, and usage since the previous interview.

“What were you doing at Spotify? I actually did a lot of different things at Spotify, but my main contribution was I built a music recommendation system, the first version.”

Claim: Erik built the first version of Spotify's music recommendation system.

“I, I joined after school in two thousand eight, two thousand nine maybe, uh, and spent seven years at Spotify and, and most of that time was building basically from scratch a music recommendation system and built up a small team and we launched a bunch of features like Related Artists and Discover Weekly and, and a bunch of other stuff.”

Claim: Erik joined Spotify around 2008–2009 after school and spent seven years there, primarily building its music recommendation system from scratch.

“I, I joined after school in two thousand eight, two thousand nine maybe, uh, and spent seven years at Spotify and, and most of that time was building basically from scratch a music recommendation system and built up a small team and we launched a bunch of features like Related Artists and Discover Weekly and, and a bunch of other stuff.”

Claim: Erik's Spotify team launched features including Related Artists and Discover Weekly.

“And, and, and arguably like a lot of what sort of later became the genesis of Modal was, um, you know, just, just building AI applications and realizing, like, the infrastructure fucking sucks. Like [chuckles] there's gotta be a better way to do that. So a lot, a lot of it sort of, you know, the rationale for, for Modal was like kind of building a tool I always wanted to have throughout my years at Spotify.”

Claim: Erik says that the poor infrastructure he encountered while building AI applications helped inspire Modal.

“Talk about it separately. But I was the CTO for, for five years at Better- Yeah -and grew the tech team from, I think, you know, two to three, four hundred.”

Claim: Erik served as Better.com's CTO for five years and grew its technology team from about two or three people to 300–400.

“So, uh, learned a lot about management and building teams and hiring, and also learned a lot about developer productivity, thought a lot about tools and things like that. Did a lot more like full stack stuff.”

Claim: Erik says his work at Better.com taught him about management, team building, hiring, developer productivity, and developer tools.

“we, uh, yeah, we, we started building it in, in two thousand fifteen, and I spent five years, uh, growing that company and then, you know, eventually turned out that doing mortgages is pretty hard and, you know, interest rates went up very quickly after the pandemic and, and that, that caused a lot of issues at the company.”

Claim: Erik says rapidly rising interest rates after the pandemic caused significant problems for Better.com.

“So, okay, you left before you had the idea for Modal, or you left and then you started idea-ing? I, I-- It's hard to say. I, I had a lot of random ideas when I left Better and, and, and I, I, I actually wasn't sure if I wanted to start a company. I was kind of just, you know, during the pandemic, I, I sort of almost took like a sabbatical. I had two kids. I, I mean, I still [chuckles] have two kids, but, uh, I, I mean, at that time, they were, they were, I think two and four, and so I decided to hang out and took some time off and was kind of working like part-time, like hacking on certain things and wasn't sure.”

Claim: After leaving Better.com, Erik was uncertain about starting a company and took time off during the pandemic.

“Maybe I should do some consulting. And I started building stuff, and I just got like super pumped about this idea that, that eventually became Modal. But actually, I built-- You know, Modal was like the, I think the third thing [chuckles] I built that I got, you know, that... So, so there were like a few attempts before then that, that went nowhere.”

Claim: Erik says Modal was the third project he built after leaving Better, following several unsuccessful attempts.

“But, but Modal was like the first idea I was like, "This is kinda cool. Like, I should do this. This is-- Like, this could be a really cool platform."”

Claim: Erik considered Modal the first of his post-Better ideas that seemed promising enough to pursue as a platform.

“Then I, I started building-- So, so maybe actually to rewind a little bit context, like when I worked at Spotify, I released an open source framework called Luigi, which is like workflow orchestration.”

Claim: While at Spotify, Erik released Luigi, an open-source workflow orchestration framework.

“So it's sort of the same category as Airflow and, and Prefect and Daxter, sort of dated category to some extent.”

Claim: Luigi belongs to the workflow-orchestration category, alongside Airflow, Prefect, and Daxter.

“Like, people don't talk about it as much, but it-it-- for a while it was like, you know, got a lot of attention. Uh, anyway, I, I never turned it into a business.”

Claim: Erik did not turn the Luigi open-source framework into a business.

“Uh, but a couple of years later when in twenty twenty-one, when I was thinking about business ideas or, or things to work on, I, I kinda came back to workflow orchestration and I started hacking on a bunch of ideas of, you know, running code in the, you know, like basically orchestrating code running in the cloud.”

Claim: In 2021, Erik revisited workflow orchestration and explored ways to orchestrate code running in the cloud.

“And what I realized was like, you know, at the end of the day, like a core puzzle piece is not just like orchestrating the compute, it's like executing the compute. And I realized like that's kind of a bigger problem actually to solve.”

Claim: Erik concluded that executing compute is a core problem beyond merely orchestrating compute, and that execution is the larger problem to solve.

“But, but like, you know, I never got back to it, long story short, and, and, and ended up building Modal, which is like the, the focus was more like, let's just figure out a way to make it easy to run code in the compu- i-in the cloud. And, and the focus was- Initially, like fairly general purpose, and it actually in a way, like it's always we've kinda gone full circle more recently. But, but we, we focused on so many different applications. It's kind of the idea was like, let's just make it super easy to run code in the c- in, in the cloud, could be GPU code, could AI, machine learning, could also be like data stuff.”

Claim: Modal initially aimed to make running code in the cloud easy across GPU, AI, machine-learning, and data workloads.

“Realized super early that like in order to do that, we'd have to go very deep in the stack and build our own file system, build our own container runtime, uh, and, and did a lot of, you know, solved a lot of hard engineering challenges. And then you mentioned ChatGPT.”

Claim: Modal concluded early that it needed to build foundational infrastructure, including its own file system and container runtime.

“That was actually not quite the killer app, but it like, if you remember like that fall, another thing came out around the same time, which was stable diffusion. And stable diffusion turned out to be a killer app for our, for what we were building at that time. We were only four or five people at that time.”

Claim: Stable Diffusion became a key use case for Modal's product at the time, whereas ChatGPT was not quite the killer app.

“So, uh, we started seeing a lot of people coming to us and wanting to run a stable diffusion model. And then we real- that, that, that made us realize like maybe gen AI inference- Mm. -is actually a really good use case for us. And so- How were you finding it? I'm this like compulsive tweeter.”

Claim: Customer demand for running Stable Diffusion led Modal to identify generative-AI inference as a promising use case for its platform.

“So I, I think I had a little bit of a following, and so, you know, in the early days, I like a lot of what drove the growth was just like me just compulsively just like tweeting about Modal and like what I was building and getting people excited and writing long blog posts about where the cloud is going and all these things.”

Claim: Erik says his early social-media posts and blog writing about Modal and cloud computing drove much of Modal's initial growth.

“Yeah. And so, you know, like work it, it's, it's still like, you know, I think a lot of our growth is we're just kinda good at social media and just like being, you know, online a lot. Nice.”

Claim: Erik says Modal's growth continues to benefit from its social-media skills and frequent online presence.

“I actually self-funded it for a while, uh, even with up until the first few employees, uh, which in a way was like kinda nice 'cause [chuckles] it made me really like frugal about money and like wanting to see like ROI.”

Claim: Erik self-funded Modal through its first few hires and says this made him especially frugal and focused on ROI.

“I wasn't taking out a salary for a while, for a long time actually. At some point it, it became a problem, uh, because I needed the money.”

Claim: Erik says he took no salary from Modal for a long time, which eventually became difficult because he needed money.

“But we didn't raise for a while and, um, uh, until early twenty twenty-two we raised a seed round. So, so at that point I'd already been working on it for about a year.”

Claim: Modal raised its seed round in early 2022, roughly one year after Erik began working on the company.

“But then it took until like basically like late twenty twenty-two for us to see any meaningful, uh, revenue. And, and- I guess that's when ChatGPT came out. Yeah, and, and stable diffusion then- Yeah...”

Claim: Modal began generating meaningful revenue in late 2022.

“in, in our case, uh, wa- was the main thing that sort. So, so we went from like, you know, hundred dollars a month in like, I think like November twenty twenty-two, and the next month we did like two thousand dollars a month, and then like January twenty twenty-three, we did like thirty thousand all of a sudden. And, you know, so, so finally I felt this like massive just like anxiety thing off my chest. I'm like, maybe this is actually like we're onto something.”

Claim: Modal's monthly revenue increased from about $100 in November 2022 to $2,000 in December 2022 and $30,000 in January 2023.

“It feels like we're onto something. And then twenty twenty-three we grew very fast, and I guess we've been growing very fast since then, basically.”

Claim: Modal grew rapidly in 2023 and, according to Erik, has continued to grow rapidly since then.

“It's just [chuckles] one and I don't, I don't know if I ever talked about like who that was and if they even know it, but I think they suspect it. It was a company in, in New York called Mage. They're doing like a stable diffusion company. So at some point, you know, in, in, in the first it, I think first quarter of, of twenty twenty, uh, three, I think they were like eighty, ninety percent of our revenue.”

Claim: Mage accounted for approximately 80–90% of Modal's revenue in the first quarter of 2023.

“But, but then after that, you know, it started diversifying a little bit. Yeah. And late twenty twenty-three we, we got Suno as a big customer. They were a big chunk of our revenue for a while too. But then, you know, it's over time, like we actually have really good, uh, revenue diversification now. Yeah. Yeah. For the stable diffusion stuff, I remember using it for like ComfyUI.”

Claim: After early 2023, Modal's revenue diversified; Suno became a major customer in late 2023, and Erik says Modal now has strong revenue diversification.

“I'm assuming a lot of people u- were using ComfyUI, like running ComfyUI workflows on it. Some people were, but, but I, I think the people that were using Modal, they tended to be more like low-level stuff. Like they did like complicated stuff. What they like with Modal is like it's always been pretty hackable. Mm.”

Claim: Erik says Modal's users often built complex, lower-level custom workflows and valued the platform's hackability.

“So they, they deploy their own custom workflows with adapters and custom inference and, and a lot of optimizations and, and all the, all the stuff we had done in order to get the developer experience to be really good turned out to be really good for also running production workloads. Uh, c- 'cause auto-scaling turns out to be actually a very similar problem, like you wanna scale up containers up and down very fast.”

Claim: Modal's developer-experience infrastructure also proved effective for production workloads, including custom inference and fast autoscaling.

“In general, like my thesis is like, you know, every new sort of era of like, you know, technology kind of runs up against limitations of, of tools.”

Claim: Erik's thesis is that every new technological era runs into limitations in the tools available to it.

“So, so I, I think, you know, i-it's of-often the case that tools exist, but, but they're often not like particularly good. And, and, and often you, you kinda need to rethink a lot of those tools in order to reap the benefits of, you know, and, and really s- you know, turbocharge and start building for this new world.”

Claim: Erik argues that existing tools are often inadequate for a new technological era and must be rethought to fully realize its benefits.

“And, and you saw this with cloud, you know, we had to invent things like Kubernetes and Docker and things like that. And, and in a, in a similar way, you know, like with, with gen AI, you could build these things on the sort of prior generation of tools. Like you, you can, you can build these things on top of Kubernetes and Docker and, and these things. But, but, but those technologies are built for a very different type of workload. They're built for a very sort of steady work, steady backend workloads.”

Claim: Although generative-AI systems can be built on Kubernetes and Docker, Erik says those technologies were designed for steady backend workloads.

“And, and you saw this with cloud, you know, we had to invent things like Kubernetes and Docker and things like that. And, and in a, in a similar way, you know, like with, with gen AI, you could build these things on the sort of prior generation of tools. Like you, you can, you can build these things on top of Kubernetes and Docker and, and these things.”

Claim: Erik cites Kubernetes and Docker as tools created for the cloud era.

“They have an even, you know, steady number of requests, like running at pretty low utilization 'cause CPUs are normally pretty cheap, and so people don't really ner- worry so much about cost.”

Claim: Erik says traditional backend systems typically handle steady request volumes at low utilization because CPUs are relatively inexpensive.

“And, but now you're entering this new world, and you have, you know, very expensive GPUs. You wanna manage capacity and, and utilization. You have very large models that you need to load over network into CPU, into GPU, and, and it, and it turns out like a lot of these tools like, you know, like they're, they're just not built for that.”

Claim: Erik says AI infrastructure must manage costly GPUs and utilization while loading large models over the network into CPU and GPU memory.

“So, so we felt that, you know, we, we had to rebuild a lot of that, you know, building our own, I, I mentioned the file system, but we also spent a lot of time over the last few years like building, you know, some really cool stuff around like CPU snapshotting, GPU snapshotting, and auto-- we built our own auto-scaler and scheduler and, and, and, and all kinds of capacity management tools. And we kinda feel like you have to do that because like you can't build, you know, for this new generation of, of software- On top of— You can't build, like, a wrapper on top of the old stuff in order to, to do the new stuff. You kinda need to start over a little bit from scratch. So I guess, like, the auto-scaling stuff-- Like, did you, did you see any, like, um, weird behaviors on the platform?”

Claim: Erik argues that new-generation software cannot be adequately supported by merely wrapping old infrastructure and requires rebuilding some components from scratch.

“So, so we felt that, you know, we, we had to rebuild a lot of that, you know, building our own, I, I mentioned the file system, but we also spent a lot of time over the last few years like building, you know, some really cool stuff around like CPU snapshotting, GPU snapshotting, and auto-- we built our own auto-scaler and scheduler and, and, and, and all kinds of capacity management tools.”

Claim: Modal built its own file system, CPU and GPU snapshotting, autoscaler, scheduler, and capacity-management tools for newer AI workloads.

“Like, was there anything that, like, broke while you guys were scaling? I mean, capacity has always from time to time been, been a challenge. Like, how do you get, like, you know, hundreds and thousands and tens of thousands of GPUs? So that's, that's always challenging.”

Claim: Erik says obtaining sufficient GPU capacity, from hundreds to tens of thousands of GPUs, has been an ongoing challenge for Modal.

“We had to go multi-region very early, so I think that was, like, kind of a fateful decision in hindsight. Like, doing that and kind of just biting the bullet and building it.”

Claim: Erik says Modal's early move to a multi-region architecture was a consequential decision in hindsight.

“Like, we always thought of, like, our, our capacity management as, like, one big pool. Like, it's like one big pool o-over the, the entire world. Like, we don't think of it as, like, separate clusters running in each region. And so that, that was, like, one of the, like, very consequential decisions we made early to kind of think about it that way.”

Claim: Modal manages capacity as a single global pool instead of separate regional clusters.

“So I don't know if there's, like, been, like, you know, specific issues. I mean, we've obviously had bugs and, you know, outages. Yeah. And I hate, you know, as much as I, you know, would love for Modal to be a hundred percent uptime, you know, and, and I think recently we've basically hit four or five nines. But throughout the history of Modal, of course, we've had, you know- Yeah...run up against scaling issues- Yeah, yeah...and things like that. The GPU market and... Well, actually, before that. Yeah.”

Claim: Erik says Modal has experienced bugs, outages, and scaling problems, while recently reaching approximately 99.99% or 99.999% uptime.

“Like, in, in the first few years, we did-- we got almost all the capacity pretty much from, from the hyperscalers, spot and on-demand capacity, mostly spot capacity, running in all the different regions, you know, using, uh, initially AWS, but also GCP and, and over time Azure. Uh, we, we started renting some capacity from Oracle pretty early 'cause we wanted, um, A10s and H100s, and we got really good rates on those.”

Claim: During its first few years, Modal sourced nearly all compute capacity from hyperscalers, primarily spot capacity, including AWS, GCP, and later Azure.

“Like, in, in the first few years, we did-- we got almost all the capacity pretty much from, from the hyperscalers, spot and on-demand capacity, mostly spot capacity, running in all the different regions, you know, using, uh, initially AWS, but also GCP and, and over time Azure. Uh, we, we started renting some capacity from Oracle pretty early 'cause we wanted, um, A10s and H100s, and we got really good rates on those.”

Claim: Modal started renting Oracle capacity early to obtain A10 and H100 GPUs at favorable rates.

“Uh, and, and over time, we added a couple more clouds, like doing reservations. But, but, like, still up until, like, actually end of last year, like, most of our capacity was either spot or a one-year reservations.”

Claim: Until the end of the previous year, most of Modal's compute capacity came from spot purchases or one-year reservations.

“Spot means you're just, like, renting it right there and then- Exactly...that's it. Yeah, and it's also preemptible, which we had to deal with. Um, but, but it sort of works in aggregate 'cause you can sort of just, you know, aggregate capacity from many, you know, hundreds of regions, basically.”

Claim: Modal's spot capacity was preemptible, but aggregating capacity across hundreds of regions made the approach workable.

“So that, that, that model worked really well until we started running up against... I mean, first of all, like, Modal started [laughs] growing, like, very fast, and so we needed more capacity. But fundamentally also, like, the cloud market started changing a lot. Um, so in the last year, basically, now in order to get the capacity we need, we have to work with a much larger set of clouds.”

Claim: Modal's rapid growth and changes in the cloud market led it to expand its capacity sourcing across a much larger set of providers.

“We actually work with almost twenty-five clouds at this point.”

Claim: Modal currently works with nearly 25 cloud providers.

“And in many cases, um, in order to get the capacity we want, like, we, we have to go and do, you know, three-plus year deals, and you have to pay a meaningful amount of money up front. Uh, and, and you're locking in pricing on, on a three-year term, right? So, you know, like...”

Claim: Modal often needs to make substantial upfront payments and sign contracts longer than three years to secure desired compute capacity.

“And, and, and so we had to spend the entire, you know, last six to nine months like, you know, kind of rethinking how we get capacity. We had to build a whole procurement, like, desk.”

Claim: Over the previous six to nine months, Modal reworked its capacity procurement strategy and established a procurement desk.

“Like, we have, like, four guys working full-time, basically. All they're doing is, like, procuring capacity, calling the clouds, you know, doing deals, vetting the capacity. We've had to build a whole engineering team, basically ten people just, like, managing, like, bare metal capacity from all over the world and dealing with network issues and bad drives and GPUs falling off the bus and, and all these things. We don't own capacity.”

Claim: Modal has about four full-time capacity procurement staff and a ten-person engineering team handling global bare-metal capacity and operational issues.

“Hmm. We rent all the capacity so far, um, you know, on, on, on typically these three-year deals. And that's, like, a big decision. Like, should we buy? Should, should we you know, you know, or should we keep renting?”

Claim: Modal currently rents all its compute capacity, usually through three-year contracts, while considering whether to purchase capacity instead.

“I, I'm actually, like, not, like, mega dogmatic about it. I, I think renting is nice because there's a certain amount of capital efficiency to it.”

Claim: Erik is open-minded about buying versus renting compute capacity and considers renting capital-efficient.

“If you're buying, you know, like, uh, I mean, first of all, there's an obvious, like, operational complexity of it. Like, you have to, you know, deal with vendors and suppliers and, and, you know, power and, and, and cables and, and all kinds of stuff. But there's also, you know, financial complexity to it. Like, how do you fund it?”

Claim: Purchasing compute infrastructure entails operational work with vendors, suppliers, power, cabling, and related equipment.

“Like, you have to, you know, deal with vendors and suppliers and, and, you know, power and, and, and cables and, and all kinds of stuff. But there's also, you know, financial complexity to it. Like, how do you fund it? Like, should you raise equity, you know? And, and you can raise a billion dollars and, and go buy hardware for it. That's a very diluted way to do it. Um, you can raise debt, but, you know, if you raise debt, you know, you, you typically wanna have, like, locked in long-term revenue to, to, to pay the interest rates. So that's also not trivial.”

Claim: Erik says hardware ownership creates financing challenges: equity financing can be dilutive, and debt typically requires predictable long-term revenue to cover interest.

“But, you know, like, we look at the economics, we look at the, the, um, the operational complexity, we look at the risks. Uh, it's a complex trade-off, and, and we're open to, to, to, to buying. We-we're sort of...”

Claim: Modal weighs the economics, operational burden, and risks of owning compute capacity and remains open to purchasing it.

“Like we're, we're-- It's-- To get the capacity we need for, for twenty twenty-seven, it's gonna take a lot of work on, on, you know, s- going a lot of different avenues to, to, to lock it up.”

Claim: Modal expects to pursue multiple approaches to secure the compute capacity it needs for 2027.

“And I, I do think over time... I mean, there, there is a lot of, you know... These are very expensive assets, uh, the hardware and, you know, I think... I mean, it's kind of staggering. You're starting to see, like, AI investments start to become, like, several percentage points of, of, of GDP of the US and, you know, starting to see, like, interest rates in, in America shooting up in the last few weeks and...”

Claim: Erik says AI investment is approaching several percentage points of U.S. GDP and that U.S. interest rates had risen in the preceding weeks.

“And so, yeah, we, we look at that and, you know, we look at all the options and, and it's possible we might have to do similar things ourselves. I wouldn't want to. Like, I would rather focus on being, you know, like a very lean software company, like doing the things we're good at. I don't think it's good in general for software companies to own hard assets, you know, in, in the same way as, like, should you own your office? Like, I, I don't think that's necessarily the right, uh, solution for, for most companies. But of course, like, in order to get the capacity we want, like, you know, everything's on the table. Yeah.”

Claim: Erik prefers Modal to remain a lean software company and generally opposes software companies owning hard assets, but says all options remain possible if needed to secure capacity.

“And I guess the supply is going to be constrained for a while, so you kinda have to do that. Like, do, do you agree with that, or do you think this is just, like, a moment in time? I don't necessarily agree with that.”

Claim: Erik does not necessarily believe GPU supply will remain constrained for an extended period.

“I mean, like, Nvidia is delivering, you know, several hundred thousand Blackwells, you know, a month, right? And, and I, I don't wanna say, like, the market is, like, a hundred percent efficient, but like, you know, in general, like, if you're willing to pay a high price Uh, you can get GPU capacity, right?”

Claim: Erik believes buyers can obtain GPU capacity if they are willing to pay a sufficiently high price.

“Like, if you're willing to pay more than everyone else and, and of course, Anthropic and OpenAI is buying up as much as they can and... But fundamentally, like there's constantly supply coming on the market.”

Claim: Erik says OpenAI and Anthropic are buying heavily, but GPU capacity continues to come onto the market.

“And, and so I don't, I don't necessarily think it's like a fully cornered resource. Prices are going up and, and I do think it's worth kind of reflecting on that as like the normal state [chuckles] of hardware is like, it's like kinda like gravity. Like, eventually hardware prices always go down.”

Claim: Erik does not consider GPU capacity a fully cornered resource and says hardware prices tend to fall over the long term.

“So we're a little bit in like a weird sort of period of time where like hardware prices are going up and, you know, it, it's hard to say, you know, h-how long this is gonna last, you know? Could be another six months, could be another thirty-six months. Like, it's really hard to, to, to know how, how, how long and...”

Claim: Erik says GPU hardware prices are rising in an unusual period whose duration is uncertain and could be six to 36 months.

“But, but I, I fundamentally don't know if like it's, it's true to say like, you know, you need to buy capacity in order to...”

Claim: Erik is uncertain whether buying compute capacity is necessary to secure access to it.

“But that being said, like, you know, capacity is gonna be tight for a while and, and you know, prices are going up and, you know, luckily I think, you know, it's not even necessarily bad for us if prices go up 'cause, 'cause you know, scarcity also means like we can charge more from our customers and, and, you know, and the fact that we have a lot of reservations locked in at a rate means like to some extent we're effectively like long GPU prices already.”

Claim: Erik expects compute capacity to remain tight and prices to rise for a while; he says scarcity may allow Modal to charge more and its reservations give it some exposure to rising GPU prices.

“Um, so um, I, I don't necessarily worry too much about prices going up. I do worry more about actually prices collapsing. I don't think that's gonna happen anytime soon.”

Claim: Erik is more concerned about a GPU price collapse than further price increases, but does not expect a collapse anytime soon.

“Uh, but I think that would be a quite bad thing for a lot of the players in the market. Why would that be bad? I think it would be bad for if, for anyone who's locked in a lot of supply.”

Claim: Erik says a GPU price collapse would harm companies that have committed to large amounts of capacity.

“Like I don't, I don't think it's gonna be like, you know, it's gonna happen anytime soon, but like, you know, like I said, like hardware prices in the very long run, they tend to go down and so it's, it's worth, you know, reflecting on that, right?”

Claim: Erik does not expect GPU prices to collapse soon, while maintaining that hardware prices generally fall over the very long term.

“Right, like right now I think, I think your, your capacity is constrained and, and that leads to sort of, you know, in economic terms there's a scarcity rent, right? Like who captured a scarcity rent? I, I think that primarily goes to two places, mostly NVIDIA, but uh, of course, but also to some extent Neoclouds. Like Neoclouds and in general like, you know, anyone owning capacity has better economics. Like that's undeniably true today with capacity being constrained. Long term, you know, in the very long run, I don't think that's necessarily entirely true, right?”

Claim: Erik believes NVIDIA and, to a lesser extent, neoclouds capture most of the scarcity rent from constrained compute capacity.

“Like there's always gonna be a little bit of a premium you can get from owning GPUs, but is it worth the risk? You know, is it worth the risk like holding these like very expensive assets on your balance sheet? I'm not so sure, but you know, we're gonna see.”

Claim: Erik thinks GPU ownership may earn a premium but is uncertain whether that premium compensates for the risk of holding expensive assets.

“Like it's gonna take a few years to get through this and, and, and in the long run, you know, we might own some, we might rent some, maybe a combination, maybe do some other crazy stuff, joint ventures, who knows?”

Claim: Erik says Modal may combine ownership and rentals or use arrangements such as joint ventures or SPVs over the next few years.

“That being said, you know, NVIDIA's shipping a lot of stuff. I think also like to some extent like NVIDIA's like kind of prefer a little bit of a diversification. Like they don't want like, you know, small amount of large buyers in much in the same way as you see, you know, hyperscalers getting a little bit of a constrained allocation.”

Claim: Erik believes NVIDIA prefers a diversified customer base instead of concentrating GPU sales among a few very large buyers.

“Uh, I, I think, you know, also NVIDIA prefers to see, you know, a lot of Neoclouds and a lot of alternative clouds like getting more allocation. And I think a little bit what we see in the market is also, this is more speculation, but I, but I think we see, you know, smaller clouds that are a little bit more sort of risky to work with, being more of an opportunity for us.”

Claim: Erik speculates that NVIDIA favors allocating capacity to neoclouds and alternative clouds, creating opportunities for Modal among smaller, riskier providers.

“Like where, you know, maybe NVIDIA, sorry, Anthropic and OpenAI, my feeling is like they prefer to buy from like the, the, the tried and true. Uh, part of it is also because they have to do a lot of training runs, so they need to have like very large blocks. But there's plenty of capacity out there, you know. You can get thirty-two nodes in Armenia.”

Claim: Erik believes Anthropic and OpenAI tend to prefer established suppliers, partly because their training runs require large blocks of compute capacity.

“Uh, part of it is also because they have to do a lot of training runs, so they need to have like very large blocks. But there's plenty of capacity out there, you know. You can get thirty-two nodes in Armenia. You can get hundred and twenty-eight nodes in Indonesia. Like, you know, like we love that kind of shit. Like, you know, so, so we can go out there and like get a lot of, you know, smaller capacity all over the world, pretty good prices.”

Claim: Erik says Modal can obtain smaller capacity blocks at good prices worldwide, citing 32 nodes in Armenia and 128 nodes in Indonesia.

“But, but like we do have like a spec sheet basically. Like when we talk to vendors, the first thing we do is we send them a spec sheet. We say, "We need this number of IOPS. We need, you know, like the CPU needs to be at least, at least, you know, have, have this many, you know, whatever the pass mark scores or whatever it is. We need to see like, you know, minimum, you know, north, north-south, uh, bandwidth, uh, egress and, and ingress." Uh, so we do have a, a list of requirements.”

Claim: Modal evaluates vendors against a specification sheet that includes IOPS, CPU performance, and network ingress and egress bandwidth requirements.

“Often we try to do like an acceptance testing period where we get like one node and, and run some tests. Over time, you know, like to some extent, it also involves a little bit of a leap of faith. You know, sometimes we sign very big commitments, you know, tens or hundreds of millions of dollars, and we start working with the vendor that we still don't fully know.”

Claim: Modal often tests a vendor's capacity on one node, yet may commit tens or hundreds of millions of dollars before fully knowing the vendor.

“Like I know which vendors I like more than others, you know, like, and, and, and we sort of prefer to do additional deals with the vendors that we've seen, you know, very good reliability. Not just that also, like it, it's not just reliability, it's also are they very responsive when, when we have s- you know, support issues? Do they fix the issues we ask for? The turnaround time.”

Claim: Modal favors repeat business with vendors that demonstrate reliability, responsiveness, and quick resolution of support problems.

“Like, we could use- utilize a lot of it, right? But, like- But you can sell it off to somebody else who wants to use it, right? I think the challenge is, like, if there was, like, something in the market that happens, there are a lot of correlations that happen at the same time. If, you know, price is going down and utilization going down, and the market for reselling capacity goes... vanishes at the-- kind of the same time.”

Claim: Erik warns that a downturn could simultaneously reduce capacity prices and utilization and eliminate the market for reselling capacity.

“I'm sitting here and, like, I'm, I'm extremely aggressive, like, buying capacity every day and, like, raising money to buy, you know. So overall, [chuckles] I'm, I'm very bullish, but, like, I do wanna point out and, like, you know, like, th-there are tail risks here, and, and when those tail risks happen, like, it could be... get very bad.”

Claim: Erik says Modal is aggressively acquiring capacity and raising money for it, while acknowledging that tail risks could produce severe losses.

“And, you know, like, I wanna build a company that, that is around for, like, 20 to 30, 50 more years, right?”

Claim: Erik's goal is to build a company that remains in business for another 20 to 50 years.

“Like, and, and so, you know, I, I think we can certainly go out and, like, buy an insane amount of capacity right now and trying to just corner the market. But, you know... Do you know about the Hunt brothers? No. It's like a legendary story in trading. They tried to corner the silver market. Mm. It failed spectacularly. Um, you know, so it's-- cornering the market, I, I think in general, like, it can be, um, not, not so good idea. Um, and... but, but we are in general very, very aggressive. But, but we're, we're ramping up, you know, like, at a, at a aggressive plan that basically matches our demand growth and matches where we wanna try to get to at the end of next year in terms of run-runway or run rate, I mean.”

Claim: Erik considers market-cornering generally unwise, while Modal is aggressively increasing capacity in line with demand growth and its target run rate for the end of next year.

“I, I think the alternative accelerators are starting to get very interesting. It is, you know, ironically. Like, every, [chuckles] every year in the last five, maybe even ten years, I've been saying, "You know, like, next year it's gonna be the year of, like, non-NVIDIA accelerators," and, and then I'm always wrong. But, but I do think 2027 might be the year of non-NVIDIA accelerators finally. So we'll, we'll see if I'm wrong this year too. But... So I, I...”

Claim: Erik thinks alternative accelerators are increasingly interesting and predicts, with uncertainty, that 2027 might be the year non-NVIDIA accelerators gain prominence, despite having made similar incorrect predictions before.

“You know, AMDs are starting to get, like, really good scores on certain things. I, I think, you know, TPUs are, are obviously used very, you know, widely internally at Google, uh, seem to work really well.”

Claim: Erik says AMD accelerators are scoring well on some tasks and Google's TPUs are widely used internally and appear to work well.

“I, I don't know much about Trainium. I, I don't feel like I see quite as, uh... But, you know, the, the team is very good. I know they're working on a lot of the software stuff. You know, hardware is hard in a, in a certain way. Like, software is, like, easier, and that makes me feel like, you know, if we just fix the sort of software layer for these accelerators, like, I think there's gonna be a lot of demand for them.”

Claim: Erik argues that improving software support for alternative accelerators could unlock significant demand because hardware is difficult while software is comparatively easier to improve.

“And, and I think fundamentally, you know, with NVIDIA being such a bottleneck, like, we'll probably see, you know, more alternative accelerators. That being said, you know, there's bottlenecks further down the pipe. I mean, you know, there's, like, TSMC and ASML- Yeah... maybe, like, a deeper bottleneck to all of this anyway. Uh, but you know, long term, I think monopolies don't always last, and so I, I'm...”

Claim: Erik expects NVIDIA's bottleneck position to encourage more alternative accelerators, while noting that constraints such as TSMC and ASML remain further down the supply chain.

“I think it'd be healthy for the market to have more alternative. As much as I love NVIDIA, like, to make that clear, like, I mean, they're obviously an amazing company and with an amazing product.”

Claim: Erik believes the market would benefit from more alternatives to NVIDIA, even though he considers NVIDIA an excellent company with an excellent product.

“Um, I, I do expect to see a little bit more diverse- diversity in, in the market. As well, also things, you know, like Cerebras and Groq and these things. Yeah.”

Claim: Erik expects the accelerator market to become somewhat more diverse.

“And, you know, and, and you see them doing really well on, on certain workloads and, you know, there, there's some interesting, you know, people doing these, like, hybrid, you know, disaggregated inference where they run, you know, NVIDIA for the prefill and, and Cerebras for the decode and, and things like that.”

Claim: Erik says specialized accelerators can excel on particular workloads, including hybrid inference systems that use NVIDIA for prefill and Cerebras for decoding.

“Oh, I haven't heard of that, actually. Yeah, and you know, Groq too. Like, 'cause a lot of these things, you know, certain things are compute bound, certain things are memory bound. So you can sort of mix and match.”

Claim: Erik says accelerators can be combined according to workload needs because some tasks are compute-bound while others are memory-bound.

“Uh, and, and I think that's quite exciting. It, it, it starts [chuckles] to get very complex when you're deploying these things in practice. Uh, but, but I, I do think there's a lot of, there's a lot of, you know, opportunity in, in, in, in these sort of, you know, more complex hardware setups.”

Claim: Erik sees significant opportunity in complex hardware configurations but says they are complicated to deploy in practice.

“I mean, there's, like, a ne-- you know, and, and, and basically the answer is like, you know, and, and this might be obvious, but, like, what they do is, like, they go out to, like, you know, Anthropic or Cor-- or, or, or OpenAI or some, some big buyer or whatever, and they basically say, like, "Hey, do you wanna buy, like, five years of chips? You know, sign this contract. We give you a good rate." And then they take that contract, and then they go to, you know, Wall Street and say, "Look, we got this, like, you know, big buyer, you know. They, they committed to ten billion dollars worth of chips. Uh, we just need the money right now." And then they go and raise the money. And then they, like, you know, go to, like, Supermicro and Dell or whatever, and NVIDIA, and, like, get the hardware, and then they install it. And then, you know, now they have, like, you know, deliver it to the customer, right? So, so you have this, like, very nice sort of financial construct where you can basically sort of pre-sell things that you haven't even bought. And, and that means, you know, you can scale up extremely fast.”

Claim: Neoclouds can scale rapidly by securing long-term customer contracts before purchasing hardware, using those contracts to raise financing, and then deploying the equipment.

“And so that's what we've seen with these neo clouds is, is, is they can scale very, very well.”

Claim: Erik says the pre-sold-contract financing model enables neoclouds to scale rapidly.

“I think Modal, because we've always had a little bit more of a PLG focus and, and, and not the sort of, you know, ability to lock in long-term contracts in the same way, it, it's a little bit more challenging from a financial perspective to scale that fast.”

Claim: Erik says Modal's PLG model and weaker access to long-term customer commitments make rapid financial scaling more difficult than for neoclouds.

“Uh, even though I think, you know, in the long run, like, we'll get to the same point.”

Claim: Erik expects Modal eventually to reach a similar scale or position to neoclouds, despite scaling more slowly financially today.

“But I think that's been kind of a fascinating thing to see with these neo clouds and especially, like, CoreWeave and, and those big ones, um, been very, very savvy with, you know, how they raise money and, you know, how to, you know, think about GPU as collateral for loans and all these things.”

Claim: Erik says CoreWeave and other large neoclouds are financially sophisticated in fundraising and using GPUs as collateral for loans.

“I think it's interesting. Like, I, I think with Compute, like, it does... There are a number of challenges with Compute as an asset class. Like, it's, it's definitely not as homogenous as other asset classes. Uh, you have different network and different locations and, and, and different disks and, and CPUs and all kinds of stuff and reliability.”

Claim: Erik says compute is less homogeneous than many asset classes because capacity differs in network, location, storage, CPUs, and reliability.

“I mean, that being said, like, you can say the same thing about oil and, like, we kind of created, like, you know, relatively commoditized, you know, tradable assets out of oil. It remains to be seen.”

Claim: Erik says oil's transformation into relatively commoditized tradable assets shows compute might also be commoditized, but the outcome remains uncertain.

“Like, I do think, you know, at the scale where we are with the sort of AI, like, data center and GPU deployments, like, we do need to be a little bit smarter with, like, risk hedging and, and, and, and how we share risks and, and, you know, having more, like, sort of insurance and, and things like that.”

Claim: Erik argues that large-scale AI data-center and GPU deployments require improved risk hedging, risk sharing, and insurance.

“That being said, like, I, I'm, I'm not entirely sure if we're gonna see, you know, GPUs becoming sort of, you know, whatever the sort of Henry Hub-like, you know, Brent crude-type, like, you know, super commoditized, like, tradable asset.”

Claim: Erik is uncertain whether GPUs will become highly commoditized, benchmarked tradable assets comparable to Henry Hub gas or Brent crude.

“Then you have Modal Sandboxes. Is there anything else? I think of the, like, the two big products that we have is basically number one is Modal Functions and number two is Modal Sandboxes.”

Claim: Modal identifies Modal Functions and Modal Sandboxes as its two main products.

“So, so with Modal Functions, we, we launched that, you know, back in twenty twenty-one. It was the idea of, like, serverless functions that are ve-very easy to deploy and very easy to scale, and we manage all the capacity.”

Claim: Modal Functions launched in 2021 as serverless functions designed for easy deployment and scaling, with Modal managing the capacity.

“That turned out to be really, really good for inference, and it turned out to be really good for managing GPUs and, and running on things like that. But we actually never had, like, you know, dedicated, like, inference product or, like, a GPU product per se.”

Claim: Modal Functions worked well for inference and GPU workloads, although Modal did not offer a separate dedicated inference or GPU product.

“Like, we, we, we never thought of Modal as, like, a Neo cloud, and, and I think there's, like, a risk of, like, getting sucked in that direction and becoming a Neo cloud. Like, just selling, you know, sort of what I think was, like, rectangular, like, "Here's, like, a three-year. You get this many GPUs."”

Claim: Erik says Modal was not intended to become a neocloud and sees a risk in shifting toward fixed multi-year GPU-capacity sales.

“Modal's always been very good at, like, you deploy something, you can kind of scale it up. You have this, like, very cyclical or, like, bursty stuff. Like, we manage all the capacity.”

Claim: Modal is designed to handle cyclical or bursty workloads by scaling deployments and managing capacity.

“And the beautiful thing is, like, 'cause we have many, many thousands of customers, we can sort of aggregate that capacity and get very smooth workloads, you know, a-a very smooth aggregate demand, which we can manage very, very efficiently.”

Claim: Modal's large customer base smooths aggregate compute demand, allowing the company to manage capacity efficiently.

“Like I mentioned, we started seeing product market fit in twenty twenty-three, and now, you know, it's about two-thirds of our revenue and, and, and empowering a lot of, you know, pr-- in practice, it's mostly inference, but, but you can use it for all kinds of other stuff.”

Claim: Modal Functions generates about two-thirds of Modal's revenue and is used primarily for inference, while also supporting other workloads.

“Modal Sandboxes, we launched that in twenty twenty-three, and it's actually kind of, you know, took a while to see product market fit, similar to Functions. Why did we launch it?”

Claim: Modal Sandboxes launched in 2023 and took time to achieve product-market fit.

“We realized a lot of people we talked to were using LMS to output code. This is, like, me and Akshat talking over lunch one day. We're like, "Maybe they need a place to run that code." And, and, and realized, like, actually, we kinda have, like, a lot of the infrastructure already to just take code and run it, 'cause we've already built the primitives for, like, managing containers and isolation.”

Claim: Modal created Sandboxes in response to the insight that code-generating language models need an environment in which to run their code.

“And, and, and realized, like, actually, we kinda have, like, a lot of the infrastructure already to just take code and run it, 'cause we've already built the primitives for, like, managing containers and isolation. We already had, you know, gVisors and isolation primitive and, and, and things like that.”

Claim: Modal reused existing container-management and isolation infrastructure, including gVisor, to build its sandbox product.

“So we, we got kind of excited about that idea and, and, and Akshat wrote a, a, a prototype and launched it in basically one day or two days or something like that.”

Claim: Akshat prototyped and launched Modal Sandboxes in approximately one or two days.

“Zero-- Like, we, we saw, like, very little, like, growth. I, I don't wanna, like, trivialize it. We did, we did see some growth. Like, there were, like, some sort of, you know, growth- I mean, it was twenty twenty-three, so. Yeah. It was, it was twenty twenty-three and, like, you know, like, some people picked it up. Like, we had some early users who, who kind of enjoyed it for, for certain things. But, but, like, overall, like, it was, like, a pretty insignificant source of revenue into going into the summer of twenty twenty-three.”

Claim: Modal Sandboxes attracted some early users and growth but contributed little revenue by summer 2023.

“So two years later, we even discussed, like, "D-do we even need this product? Like, should we shut it down?" Then it just started exploding. So twenty twenty-three, I...”

Claim: Modal considered discontinuing Sandboxes before the product later began growing rapidly.

“And, and I think what, what really happened in twenty twenty-three was, like, LMS got really good at generating code. In twenty twenty-three? Twenty twenty-three. I guess, yeah. I mean, it was, like, around that time. Like, I, I feel like that was, like, when they went from, like, being able to, like, refactor functions and, like, write snippets to, like, generating, like, full apps, right?”

Claim: Erik says that around 2023, language models advanced from editing functions and writing snippets to generating full applications.

“And, and, and so we started seeing a lot of growth coming from, like, vibe coding, you know, companies like Lovable and, and other ones. Uh, and, and then, you know, companies like Ramp started using Modal for background agents. And then we're seeing-- started seeing tremendous growth coming from a number of Neo labs doing things like reinforcement learning on Modal Sandboxes. So Sandboxes is now about a third of our revenue.”

Claim: Modal Sandboxes grew through demand from vibe-coding companies, background-agent workloads, and reinforcement-learning workloads at AI labs.

“And then we're seeing-- started seeing tremendous growth coming from a number of Neo labs doing things like reinforcement learning on Modal Sandboxes. So Sandboxes is now about a third of our revenue.”

Claim: Modal Sandboxes generate approximately one-third of Modal's revenue.

“And, and to me, it's, like, really exciting 'cause, like, even back in the early days of Modal, I always knew that Modal, you know, is about, like, creating a whole toolkit of products. Like, we always wanted to be almost like a cloud, you know. You have the storage and the compute and a bunch of different ways to work with it. We always wanted to be a multi-product vendor, but that wasn't necessarily true in practice until, like, now, where, like, I think now it's, like, kinda clear. We have Functions. We have Sandboxes.”

Claim: Modal's long-standing vision was to become a multi-product cloud platform, a strategy now represented by Functions and Sandboxes.

“We're working on a bunch of other stuff too, working on training, working on Batch. We're working on a bunch of other stuff.”

Claim: Modal is developing additional products, including training and batch processing, beyond Functions and Sandboxes.

“Do you have anything in the works that, you know, you think is gonna be, like, that next product? Any predictions maybe about the market, anything like that? I mean, I'm, I'm very bullish on training. I think training is, like, kind of an underserved market. Obviously, there's, like, a lot of focus on, like, very large, you know, training runs. Uh, but we, we actually see a lot of demand in the sort of smaller and, and mid-sized training segment, so I'll, I'll be talking a little bit about that later this week. Hmm. We're organizing a conference.”

Claim: Erik sees training as an underserved market, especially for smaller and mid-sized training workloads.

“Batch is, I think, like, a really interesting market. I, I think we see a lot of demand for sort of, you know, customers that need a You know, we had a customer the other day that needed to transcribe, you know, five thousand years of audio. They had like, you know, millions and millions of audio files and, and we, we had a lot of capacity, so we, we let them do that. You know, it took a few days. I, I think there's some really interesting use cases there.”

Claim: Erik sees batch computing as an interesting market, citing customer demand for large jobs such as transcribing 5,000 years of audio over several days.

“Very bullish on, like, low latency stuff. I, I think there's gonna be a lot of, you know... Once we sort of get the audi- like, the speech-to-speech models working, like the voice models, I, I sort of expect that to be very interesting market. Then of course, like, all this, like, agentic stuff, right?”

Claim: Erik expects low-latency applications, including speech-to-speech and voice-model use cases, to become an interesting market as the models improve.

“In, in a way, like, I, I have, like, mixed feelings about, like, the word agentic 'cause I feel like in, in certain ways, like, I think all software will become agentic, so it's almost like, to me, like, what does it mean to be agentic? Like, I think, I think all software in a certain way, like, will become, like, AI-powered over, over time.”

Claim: Erik thinks software will increasingly become AI-powered and agentic, making the term “agentic” less clearly distinguishing.

“But, but, like, clearly there's, like, a set of, like, building blocks of, like, you know, doing more sort of LLM-powered software that seems to be sort of, you know, common across all the different use cases. And, and we're, we're, we're like, you know, the way I think about models, like, we're in the business of building building blocks. You know, we, we wanna give you, like, amazing, generalizable, battle-tested, you know, general purpose, uh, composable building blocks and, and I think there's some really interesting stuff also, like, you know, with the, with the agent use cases.”

Claim: Modal aims to provide general-purpose, composable infrastructure building blocks for the common patterns in LLM-powered software.

“Like, what changed in the market? I mean, we, we've been building a training product for multiple years, so- Oh, okay... you know, on and off. So, so we, we-- I, I wouldn't say we're like we've never been focused on it. But it's like, it's like one of those things where, like... I mean, now we're like a hundred and fifty engineers but, you know, a year or two ago we, we were a much smaller number of engineers and so... But just virtue of, you know, being smaller, we had to focus a little bit more, and so inference became, like, our sole focus and, and was our sole focus for many years. Yeah. But we're bigger now.”

Claim: Modal has worked intermittently on a training product for years, while focusing exclusively on inference during its smaller-company period.

“Post-training reinforcement learning, I think there's some really exciting stuff around that. Um, I, I think there's also many other, like, stuff around, like storage and building data warehousing and feature stores and real-time streaming pipelines and, and all kinds of stuff like... I, I don't know.”

Claim: Erik sees potential in post-training reinforcement learning and may pursue infrastructure products for storage, data warehousing, feature stores, and real-time streaming.

“I, I think the long-term view of, of Modal is, like, basically like, you wanna build some cool AI shit, like, you know, we got you covered. Like, you know, we got all the building blocks. Like, come to us and, like, we'll help you, you know, take all these building blocks and compose them, and that's the beauty of software is, like, you, you can, you know, build almost anything if you have the right building blocks. So that, that's very much the sort of company and the product I wanna build. When you say training, are you focused on more of, like, the bigger enterprises doing huge training runs? No. Or are you- Okay. No.”

Claim: Erik's long-term vision is for Modal to offer a broad toolkit of composable building blocks for building AI applications.

“They're gonna go and, like, you know, do these, like, you know, big hyperscaler or, or rather neo- neo-cloud mega deals and get, you know, a hundred thousand GPUs or whatever. But, but, but there is, like, a little bit of an underserved market of, like, in my opinion, you know, like, researchers iterating or, like, people training smaller models.”

Claim: Erik considers researchers and teams training smaller models an underserved training market, distinct from hyperscaler-scale training deals.

“There's, you know, text to speech and speech to text models. Computational biotech is actually a market that we see a lot of demand from. We have a lot of customers in that space training relatively s- you know, modest models. You can train on sixty-four GPUs or whatever. Recommendation system. A lot of the sort of traditional machine learning. There, there's a lot of things where, like, you don't need a, you know, a hundred thousand GPUs. That market I think is like...”

Claim: Modal serves demand for modest-scale training in computational biotech, speech, recommendation systems, and traditional machine learning, including jobs using around 64 GPUs.

“We're, we're not gonna, like, focus on that for a while, but, but I think there's a lot of stuff in the, you know, slightly smaller markets that, that, that are relatively underserved.”

Claim: Modal does not plan to focus on smaller training markets for a while, despite Erik's view that they are underserved.

“I feel like fine-tuning the value prop a couple years ago was like, "Yeah, you know, open source models, they're not quite as good, but, you know, if you take, you know, a small model and you fine-tune it on your specific domain, you can actually reach the frontier in terms of, you know, quality and cost." Uh, today I, I would say, like, the open source models are so good that, like, you can basically just, like, swap them out and just run, you know, instead of, you know, the, the frontier ones, you can run, you know, Chemica three or, or, or, or whatever, DeepSeek four point one or GLM five point three, and they're, like, basically almost as good as, like, the frontier model at a much lower cost.”

Claim: Erik says open-source models have improved enough that customers can often replace frontier models with nearly comparable models at substantially lower cost, reducing the need to fine-tune.

“So we still see a lot of demand of, uh, for, for post-training, and I do think it's a really interesting space, and we're working on some, some products in that space. You know, both sort of SFT and, and, but also reinforcement learning and things like that. I would also say, you know, fine-tuning is no longer the thing that you kinda have to do. Like, it, it's, it's very optional.”

Claim: Modal sees continued demand for post-training and is developing products for supervised fine-tuning and reinforcement learning.

“You know, both sort of SFT and, and, but also reinforcement learning and things like that. I would also say, you know, fine-tuning is no longer the thing that you kinda have to do. Like, it, it's, it's very optional. Like, you, you can... Open source models, like, work really well just off the shelf too. Back to sandboxes, actually. I forgot to ask you.”

Claim: Erik believes fine-tuning is no longer necessary in many cases because open-source models can perform well without modification.

“I, I think the thing we've always been very, very good at, I mean, we're infra geeks, right? Like, we're very good at, like, you know, scaling and reliability and, and doing a lot of stuff. And, and I think that's why... That explains to a large extent, like, why we have, you know, in, in my opinion, I don't have numbers, but, like, I think we have by far the largest market share in this space.”

Claim: Erik attributes Modal's sandbox position to its infrastructure, scaling, and reliability expertise and believes it has the largest market share, while acknowledging he has no market-share data.

“Like, we're just very good at running, you know, hundreds of thousands or even millions of sandboxes. Yeah.”

Claim: Modal's systems can operate hundreds of thousands to millions of sandboxes.

“I will say we'll have to-- We've had to rewrite our systems, you know, a bunch of times in order to handle that scale. So, so... But we were always, like, very good at it. Th- there are other things, like, over time.”

Claim: Modal has repeatedly rewritten its systems to support its sandbox workload scale.

“I mean, the sandbox space in, in general, like, you know, like, it's, like, not the most, like, differentiated space but, you know, we're working on some really cool stuff.”

Claim: Erik considers the sandbox market generally undifferentiated but says Modal is working on differentiated capabilities.

“Like, over time, like, we've had to add support for VMs, like virtual machines, so we, we, we previously built our, our sandbox product on, on gVisor, but now we offer full virtualized machines as well. Turns out a lot of... For, for certain use cases you kind of need full Linux compatibility.”

Claim: Modal expanded its gVisor-based sandbox product to include full virtual machines for use cases requiring complete Linux compatibility.

“There are other things we've had to build, you know, better primitives for, like, more interactive, you know, piping data back and forth between sandboxes, uh, working with file system. We've had to spend a lot of time on snapshotting.”

Claim: Modal has developed interactive data-piping and file-system primitives for sandboxes and invested substantially in snapshotting.

“I think a lot of the things that over-- in the long run will differentiate will be, like, you know, really cool things like, you know, forking sandboxes, snapshotting, resuming snap, uh, like, live migrations.”

Claim: Erik expects sandbox forking, snapshots, resumption, and live migration to differentiate sandbox platforms in the long run.

“I- in general, like, I, I think sandboxes The novelty of sandboxes was always, you know, this sort of ephemerality. Like, you can spin up sandboxes for a few seconds and, and, and that's something I, I... Like, that's a string I, I wanna pull more in.”

Claim: Erik views short-lived, rapidly created sandboxes as a core feature and wants to make that ephemerality more central.

“I, I think there's a lot of really cool stuff you can do, you know, kind of decoupling the agent and, and the harness, decoupling it from sandboxes, having agents control many sandboxes, forking sandboxes, you know, snapshotting things. Like, you know, like, running basically, like, very short-lived, may-maybe one per command, like things like that. So I, I think there's a lot of cool stuff there. We've already spent a lot of time doing that, you know, around snapshots. Uh, but, but I think there's a lot of cool stuff, uh, to be, to be built there as well. You brought up snapshotting a couple times.”

Claim: Erik expects agent systems to benefit from separating agents from execution harnesses and using many short-lived, forkable, snapshot-capable sandboxes.

“I think the, the big thing for me is, like, today a lot of sandbox workloads are kinda like lift and shift. Like, people think a lot about sandboxes almost like, you know, "I have my dev box. Now I'm gonna shove it in the cloud. I'm gonna have an agent control it. Maybe I'll run the harness inside the sandbox."”

Claim: Erik says many current sandbox workloads simply move a developer's existing environment to the cloud for an agent to control.

“I don't know if, like, long term that's, like, the right way to think about, you know, on, like, compute for agents. I, I think a much better model would be you're gonna have these, like, very long-running agents whi-which almost, like, to me, are, like, kind of durable execution workflows. Like, they're gonna run, you know, twenty-four/seven. You know, every once in a while they're gonna run commands and, and, you know, having the ability to, like, spin up a sandbox in, you know, a hundred millisecond, run a few commands, and then, like, shut it down and, like, snapshot it.”

Claim: Erik doubts that moving persistent development boxes into the cloud is the best long-term agent-compute model and favors durable agents that launch short-lived sandboxes for commands and snapshot them afterward.

“I think the faster you can get that, like, you know, wake up, you know, go to sleep cycle, the more you can sort of create sandboxes that only run for, like, a very brief period. And, and that will make compute much more efficient, and it will also enable you to, to, to, to do some really cool things, you know, like forking and, and moving things around and, and, and, you know, having...”

Claim: Reducing sandbox startup and shutdown latency enables shorter-lived workloads and more efficient compute use.

“Like, imagine, you know, like, Claude Code instead of, you know, having sort of a very linear flow, like sequential flow, like kind of branching and, you know, running hundreds of agents at the same time.”

Claim: Erik envisions coding agents such as Claude Code branching into hundreds of concurrent agents instead of executing tasks only sequentially.

“I, I think a lot about, like, CI as, like, another, like, illustrative example. Like, for most of the, the history of CI, uh, you know, talk about GitHub actions and things like that, like, you had a very sequential flow. Like you, you, you know, you slowly b- install a bunch of stuff, then you run the test suite, and then, you know, generate some reports or whatever. And it's like you can break it up into shards, but it's kind of annoying. Like, what I would love to do is, like, take your entire test suite and just run every single test in its own sandbox, right? Like, and just paralyze the shit out of that. Run, like, a thousand sandboxes. Why can't that be done?”

Claim: Erik envisions CI systems running each test in a separate sandbox and parallelizing the test suite across large numbers of sandboxes.

“I mean, the problem is, like, there's too much, there's too much overhead around, like, you know, starting and stopping sandboxes.”

Claim: High sandbox startup and shutdown overhead currently limits frictionless, large-scale parallel workloads.

“So I think the more, like, you, you create, like, you know, frictionless, like, starting and stopping, the more you can enter a world where, like, you can do this, like, extremely bursty, like, large scale, like, run as much as you think want in parallel.”

Claim: Frictionless sandbox startup and shutdown could enable workloads to run in parallel at large scale and in short bursts.

“You know, once we figure out all these primitives and, and the right building blocks, I think it's gonna unlock a lot of really cool stuff that, you know, we, we previously could not do with, with software.”

Claim: Erik expects better infrastructure primitives and building blocks to enable software capabilities that were previously unattainable.

“Like, the overhead of parallelization right now is very high. 'Cause, like, right now, like, in GitHub actions for instance, like we, we parallelize in, in the Modal, the test suite, right? But, like, what that basically means is, like, now if you take your test suite and you split it up in two, like, now each of those shards both need to do all the installs and, and the test setup. Mm. And that takes a long time. Like, it, it can take minutes. But let's say you can do the test setup once, and then you snapshot the container, and now you clone it, and now you can run the test suites independently.”

Claim: Current CI parallelization has high overhead because each test shard repeats installation and setup, potentially taking minutes.

“And that takes a long time. Like, it, it can take minutes. But let's say you can do the test setup once, and then you snapshot the container, and now you clone it, and now you can run the test suites independently.”

Claim: Snapshotting a prepared test container and cloning it can let tests run independently without repeating installation and setup.

“I think a lot of, a lot of the, the benefits cannot be, uh... Y-you, you can't materialize those benefits un, un-until you get the, the friction and, and the overhead down. Mm. The transaction cost.”

Claim: Erik says the benefits of large-scale sandboxing and parallelization depend on reducing their friction, overhead, and transaction costs.

“I mean, the, the managed agents with, with Anthropic, uh, there's a sort of, uh, bring your own, um, sandbox, uh, integration, so you c- you can use Modal for those, yeah. Oh, cool. How did that come a- like, come to be?”

Claim: Anthropic's managed-agent offering allows customers to bring their own sandbox, including Modal Sandboxes.

“I think Anthropic is, like, trying to create, you know, like, a thriving ecosystem and community. Mm. And, and so I think, you know, they make money from inference, right [chuckles]? Like, so- Okay, yeah... I, I, I think it's very much in their interest to promote, you know, like, uh, for everything else, you know, as much of, you know, of an ecosystem as possible, and they wanna have the largest possible surface area, as long as you use Anthropic for inference. [chuckles] So I, I think it makes, like, very rational sense for them to, to, to partner with, with us- Okay... as a sandbox provider. Also, I feel like we have to bring up the Hugging Face incident.”

Claim: Erik believes Anthropic promotes a broad ecosystem and wide integration surface because it benefits when customers continue using Anthropic for inference.

“And so, you know, just to comment on that, like, basically the, the, the agents running inside, you know, that, that, that exploited Hugging Face, they discovered a, a vulnerability in an app that was deployed on Modal. Mm. And they used that to sort of, uh, as almost like a springboard for, for getting access, getting full internet access, is my understanding. I don't remember the full details.”

Claim: Erik's understanding is that agents in the Hugging Face incident exploited a vulnerability in an app hosted on Modal and used it to gain broader internet access.

“To be clear, this is an app running a Modal that was hacked. Like, it was not Modal that was hacked. Yeah. So it's like AWS, a lot of people run stuff on AWS. Occasionally those app get hacked. Does that mean AWS got hacked? Like, no. [chuckles] Of course not, right? And, and so there's a lot of people running apps on Modal and, you know, one of those apps got hacked, and I, I'd say...”

Claim: Erik distinguishes the Hugging Face incident from a breach of Modal itself: an application hosted on Modal was hacked.

“But, but, like, there are some lessons we can learn and, and, and I think one of the things we've been reflecting on is, like, could we have more sensible, you know, r- security restrictions by default?”

Claim: Modal is exploring stronger default security restrictions in response to lessons from the incident.

“I, I think to use the AWS analogy in the pa- uh, as just, just as another analogy, you know, S3 used to be this, like, you know... It used to be very easy to, to accidentally make an S3 bucket public, and AWS realized, like, you know, it doesn't necessarily cause a, you know, pose a security r- Problem for AWS per se. But a lot of their customers accidentally would, you know, like leak data, and so S3, it's much harder now to make S3 buckets public.”

Claim: Erik cites AWS's tightening of S3 public-access defaults after customers accidentally exposed data as an example of safer defaults.

“And so in a similar way, it's like we're exploring like, you know, can we make security a little bit more restricted by default without necessarily, you know, destroying the magic that make Mo- that makes Modal good?”

Claim: Modal is considering stronger default security controls while trying to preserve the platform's usability and flexibility.

“And I think there's also, you know, as kind of a separate sidetrack is like agents are getting really good at cybersecurity. So we've actually also, as a sort of, you know, separate sort of learning from this incident, like we started using agents internally quite a lot to, to, to try to find vulnerabilities.”

Claim: Erik says agents are increasingly capable in cybersecurity and Modal is using them internally to probe for vulnerabilities.

“So far, hasn't found anything, but like they're very good at like probing into like very deep parts of the system and exploiting-- trying to exploit things and trying to e-escalate and get out of sandboxes.”

Claim: Modal's security agents have found no vulnerabilities so far, though they can probe deep system areas and attempt exploits and sandbox escapes.

“You're just like, "Hey, like, find security vulner-vulnerabilities"? Well, no, you just like, you just put them in a sandbox and, and tell them like, "Can you try to exploit?" Just run through the- Yeah... entire repo? "Can you just try to escape from the sandbox?" Yeah. It's like escape the room, but for agents. [laughs] Cool. When did you start doing that? We started doing it a, a few months ago. Okay. And it's pretty exciting.”

Claim: For the past few months, Modal has used agents in sandboxes to attempt exploits and sandbox escapes as a security test.

“I mean, my, my view on like cybersecurity is that, you know, I actually think they're very good at finding vulnerabilities, but I think you can also use them defensively w-w-very well.”

Claim: Erik believes AI agents can identify vulnerabilities and can be used effectively for defensive cybersecurity.

“So I, I, I think, you know, companies like Modal and many other companies right now, like I think, you know, if, if you go to talk to like pretty much every CS, I'm sure they're all thinking about the same thing. Like how do we use agents defensively to, to, to, you know, fix all our vulnerabilities? So over time, actually, in a way, like I'm not that nervous. I think if, if anything, like agents and, and cybersecurity, all the breakthroughs in cybersecurity will, will make the world safer, if anything.”

Claim: Erik expects AI-enabled defensive cybersecurity and related breakthroughs to make the world safer overall.

“But when I think about like cybersecurity, a lot of what you're selling is effectively fear, and that's almost like a branding and like marketing strategy as much as, you know, obviously the core products. But I think, you know, brand and, and things like that are actually like arguably like more valuable in the era of AI. So I, I, I wouldn't necessarily jump to the conclusion that they're cooked. I think they have, you know, deep relationships with the enterprise customers, and they have, you know, big sales teams who are very good at telling, you know, spooky stories and, and things like that.”

Claim: Erik does not think AI necessarily makes cybersecurity companies obsolete, because brand and established enterprise relationships may remain valuable.

“I, I, I mean, to some extent it, it does reduce a little bit the need for us to do-- to bring in pen testers, like human pen testers, which most startups like do on a regular cycle. So maybe previously, you know, we would do it... I actually don't know what the cycle was, but, but I think we used to do it like once every six months, roughly. Maybe now we only need to do it once every year. I, I don't know. But, but like I think with all of these things, you know, humans can also use them to get more efficiently. So like, you know, like I, I, I don't know.”

Claim: Erik thinks agent-assisted security work may reduce Modal's human penetration-testing frequency from about every six months to perhaps once a year.

“I'm, I'm kind of a believer also that like humans with AI is even more powerful now.”

Claim: Erik believes human-AI collaboration is more powerful than relying on humans or AI alone.

“So like maybe there's an argument that like we can bring in pen testers who are really good at using agents and, you know, find vulnerabilities even faster. I, I don't know. I, I'm not that, you know... I'm an optimist.”

Claim: Erik suggests, with uncertainty, that agent-assisted human penetration testers may identify vulnerabilities more quickly.

“I started a company, so like- Yeah... you know, I'm, I'm generally like optimistic about everything. But like I, I certainly don't think humans are cooked or companies are cooked.”

Claim: Erik does not believe AI makes humans or companies obsolete.

“Like I, I think in the era of AI, a lot of companies are gonna figure out, you know, how to use AI to be even more efficient. Mm-hmm. I wanna transition a little bit to like developer experience.”

Claim: Erik expects many companies to improve their efficiency by adopting AI.

“'Cause like early on, I mean, even now, like you guys focus a lot on that. Yeah. I mean, you've talked a lot about that. Does developer experience even matter anymore? I think it still matters. Y-and, and I think you're, you're sort of, you know, like the context, I think you're fish-- or like the thing you're talking about is like now with agents, right? Like do, does developer ex- Yeah. To that point, though, like almost all the things that make developer experience good also makes the agent experience good, is kind of what we've seen in practice.”

Claim: Erik believes developer experience remains important because many of the same qualities also improve the experience of AI agents.

“Which is like things like having good docs, having descriptive error messages, having fast feedback loops, right? Like I do think there's-- certain things have changed.”

Claim: Erik identifies clear documentation, descriptive errors, and fast feedback as practices that benefit both developers and agents.

“The other day, I was like setting something up with Cloudflare, and it's like I find Cloudflare like notoriously annoying to work with sometimes. And, and I threw an agent at it, and it was able to like just configure a bunch of like as-- you know, things in, in Pulumi for me. And you know, and, and so I will say like agents does-- they do make it a little bit easier to work with things that are like harder to work with in the past, you know? Like the sort of human toll of dealing with YAML has arguably gone down a little bit 'cause you can just like, you know, tell your agent to deal with it.”

Claim: Erik says an AI agent configured Pulumi resources for a Cloudflare setup that he finds difficult to work with.

“And, and I threw an agent at it, and it was able to like just configure a bunch of like as-- you know, things in, in Pulumi for me. And you know, and, and so I will say like agents does-- they do make it a little bit easier to work with things that are like harder to work with in the past, you know? Like the sort of human toll of dealing with YAML has arguably gone down a little bit 'cause you can just like, you know, tell your agent to deal with it.”

Claim: Erik says agents can reduce the effort required to work with difficult systems and configuration formats such as YAML.

“I think to some extent also like infrastructure over time anyway becomes and needs to be more programmatic. I think that's always been a thesis about sandboxes, is that, you know, like things that you previously had to configure using YAML, that should just be an SDK. That should be more like imperative, like launch a sandbox, like do stuff with it. Tear the sandbox down, right? Like rather than sort of declarative using, using, you know, some sort of DSL.”

Claim: Erik expects infrastructure to become more programmatic, with sandbox configuration shifting from declarative YAML toward SDK-based imperative operations.

“It also pushes like a little bit of a different interface that maybe previously didn't exist. I think in either way, like SDKs are as relevant as they would-- a-as they always were, if not more.”

Claim: Erik believes SDKs remain as relevant as ever, if not more so.

“Uh, I mean, we, we also-- we actually do run a lot of internal tests where we have agents. We tell agents to like build a mobile app and run it.”

Claim: Modal tests its products by asking AI agents to build and run mobile applications.

“And I mean, a, a very concrete thing that is kind of funny is that we've actually seen that like because of agent training data- It's very hard to deprecate things now. Uh, we see that, like, agents often, like, love using, you know, old versions of modal syntax, and, and it's kinda hard to nudge them away from that. So, so we've actually had to be a little bit more conservative going forward, like, you know, uh, over the last year or so with, with how we deprecate or remove features.”

Claim: Because agents learn outdated Modal syntax from training data, Modal has become more cautious about removing or deprecating features.

“It takes-- We have to be very careful to update every single example that's out there online. Yeah. And, and, and, you know- You have skills, right? Yeah. There's, like, stuff like that. But, but, like, there's also an insane amount of, like, modal code that's now, like, online too that goes into the training data.”

Claim: Erik says online examples and code influence agent training, so Modal must carefully update its public examples and account for the large amount of Modal code online.

“We often, like, run internal benchmarks, and we're like ask a, an agent like, "Hey, like, can you deploy a stable diffusion for us? Use whatever tool you want." Yeah. And, and we see if it recommends modal or not, and those things are very useful for us to sort of track over time. Or like, "Hey, spin up some sandboxes," or like, "Hey, like, rank all the sandbox vendors." So, so we, we spend a lot of time, like, doing, doing those things to, to, to make sure- I mean, what do you do about that? Yeah, you say, "Hey, like, you know, build this stable diffusion thing."”

Claim: Modal benchmarks agent recommendations and task performance, including whether agents choose Modal for deployments and sandbox workloads.

“Does it recommend modal or not? Like, yeah, what do you do about that? Yeah, I, I think that's harder. I, I think, [sighs] you know, it's like search engine optimization. I think it's primarily through content. It's like you just have to put out a lot of content. Interesting. You can also, to some extent, like, sort of see the perception, I think, a little bit more. Like, you can ask an agent, like, you know, "How does...”

Claim: Erik believes improving how agents perceive or recommend a company is primarily a content strategy, analogous to SEO.

“You know, rank all the sandbox vendors by reliability," or, "Rank all the sandbox vendors by price," and, and sort of see. And it does feel like agents are a little bit more prone to, like, misc- understanding pricing and, and, you know, things like that. So we also spend a lot of time, like, just making content more clear and more, like, you know, understandable for agents. 'Cause we know they're all indexing, like, everything that's out there- Yeah...”

Claim: Erik says agents can misinterpret pricing, prompting Modal to make its public content clearer and more machine-readable.

“Uh, I think the truth is, like, that is probably something you can do. My, my guess is, like, LM training data today is probably quite insensitive to, like, what's, like, AI output in the first place- Yeah... and you can probably, like, create a lot of slop. Like, we thought about it. Like, I don't love that idea, but, like, in theory, like, much like, you know, SEO in the early days, and still to some extent, like, auto-generated content is, like, a very effective strategy.”

Claim: Erik thinks mass-producing AI-generated content could influence agent recommendations because training data may not distinguish AI output, although he dislikes this strategy.

“Like, I don't love that idea, but, like, in theory, like, much like, you know, SEO in the early days, and still to some extent, like, auto-generated content is, like, a very effective strategy.”

Claim: Erik says auto-generated content has been an effective SEO tactic historically and remains effective to some extent.

“We're in a new era of, like, building applications and, and, and, you know, things that we couldn't build, you know, just a few years ago. And we see a lot of companies that, you know, they need a little bit of help, right? Like, you know. And, and so we started investing in FDEs a couple years ago, but it, but it's really been this year we've kind of scaled it up a lot.”

Claim: Modal started building its forward-deployed engineering team a few years ago and expanded it significantly this year.

“I think we started this year having, like, four FDEs. Now I think we have, like, twenty-four or something like that. So we've grown it a lot. And they do a...”

Claim: Modal's forward-deployed engineering team expanded from approximately four people to approximately 24 during the year.

“You know, it's more-- It's not exactly the Palantir model, but they do a lot of, like, heavy, you know, sort of customer engagement. Like, they work with a lot of customers on performance tuning, especially with LMS.”

Claim: Modal's forward-deployed engineers provide hands-on customer engagement and performance tuning, particularly for language-model workloads.

“We, we see that there's so much benefit from, like, you know, sitting down with customers and training draft models and tweaking, uh, you know, parameters in DLM and SGLang. Um, they, they often, you know, fix bugs. Or, or not fix bugs, but, like, you know, when, when customers run into issues, like, they, they often work with customers trying to figure out, like, how do we work around that?”

Claim: Modal's forward-deployed engineers assist with model and parameter tuning in DLM and SGLang and help customers troubleshoot or work around issues.

“Or, uh, another thing we, we've used FDEs a lot for is, like, you know, just kind of proactively, you know, if, if we see certain customers, you know, over-provisioning with modal, like customers requesting too much CPU or too much memory, for instance, like, that's kind of expensive. And, like, we, we try to catch those things, and we try to sit down with those customers like, "Yo, like, you're spending a lot of money on modal right now. We can actually help you save money."”

Claim: Modal's forward-deployed engineers proactively help customers reduce costs when they over-provision CPU or memory.

“And then, of course, like, with, with sort of active deals, you know, we, we help them migrate code from other platforms, uh, things like that. Um, I think, you know, we, we, we never had the intention of becoming, like, a service-heavy business.”

Claim: Modal's forward-deployed engineers assist prospective customers with migrating code from other platforms.

“And then, of course, like, with, with sort of active deals, you know, we, we help them migrate code from other platforms, uh, things like that. Um, I think, you know, we, we, we never had the intention of becoming, like, a service-heavy business. I, I think the, the goal for FDEs is, like, you know, A, is to be part of, you know, the, the sort of sales process and, like, closing deals. B is to serve customers and, and do some, like, you know, customer support. But, like, I actually think that, like, the most important thing is, like, C, is, like, bridging gaps in the product that we later wanna turn into, like, real features. So I, I think to a large extent, you know, anything an FDE does, you know, my question is always, like, long term, how do we turn this into product so that we don't need an FDE for this? Yeah. Do you guys hire PMs?”

Claim: Modal's forward-deployed engineers support sales and customers, but their primary strategic purpose is to reveal product gaps that can be turned into features rather than making Modal service-heavy.

“But, like, I actually think that, like, the most important thing is, like, C, is, like, bridging gaps in the product that we later wanna turn into, like, real features. So I, I think to a large extent, you know, anything an FDE does, you know, my question is always, like, long term, how do we turn this into product so that we don't need an FDE for this? Yeah. Do you guys hire PMs?”

Claim: Erik wants customer-specific work done by forward-deployed engineers to become product features over time, reducing the need for ongoing manual support.

“So I'm, I'm actually not, like, anti-product management. I do think that a lot of companies in the past hired product managemen- product managers too eagerly, and, and I think there's a risk that you, you kind of... I can sort of see it in a lot of products. Like, certain products, like, you can tell, like- There's just, like, eager product managers that just thought of, like, kind of, like, widget-y things and just shipped them. I think doing product management at an infra company really well is very, very hard because it take, it takes, like, a very deep, you know, level of strategic thinking, thinking about, like, what are the foundational building blocks?”

Claim: Erik is not anti-product-management, but believes companies have over-hired product managers and that infrastructure product work requires unusually deep strategic judgment.

“Where do we wanna be in one to two years? 'Cause the truth is, building infrastructure is very hard. And how do we make sure we don't end up with, like, thirty-five features that are, like, kinda similar looking, but all do, like, a little bit slightly different things? That sort of level of thinking is something that, you know, is very rare to find. And then for that reason, we haven't aggressively tried to build up a product management team yet. Yeah, that makes sense. How would you say your teams have been evolving, maybe, like, since GenAI?”

Claim: Erik says infrastructure product management requires long-range strategic choices about foundational capabilities and avoiding a proliferation of similar features.

“Where do we wanna be in one to two years? 'Cause the truth is, building infrastructure is very hard. And how do we make sure we don't end up with, like, thirty-five features that are, like, kinda similar looking, but all do, like, a little bit slightly different things? That sort of level of thinking is something that, you know, is very rare to find. And then for that reason, we haven't aggressively tried to build up a product management team yet. Yeah, that makes sense. How would you say your teams have been evolving, maybe, like, since GenAI?”

Claim: Modal has avoided rapidly expanding product management because Erik considers strong infrastructure product managers difficult to find.

“In a funny way, like, yes. I, I do think team structures have changed a lot, but I'm actually not convinced it was, like, GenAI driving that.”

Claim: Erik observes changes in team structures but is uncertain that generative AI caused them.

“I, I think, in particular, like, one thing I've seen a lot is that there's been sort of a focus, like a swing back towards being in IC that I actually think is very, very positive for the space as a whole.”

Claim: Erik believes the technology sector's shift toward individual-contributor roles is positive.

“I think that shift actually started even before GenAI, but I do think it's, like, heavily accelerated with AI. Like, you know, we, we, we make it very clear to everyone at Modal that, like, manager or IC, it's like there's no difference in prestige. Like, everyone is just a member of technical staff.”

Claim: Erik believes the move toward individual-contributor roles predates generative AI and has been accelerated by AI.

“I think that shift actually started even before GenAI, but I do think it's, like, heavily accelerated with AI. Like, you know, we, we, we make it very clear to everyone at Modal that, like, manager or IC, it's like there's no difference in prestige. Like, everyone is just a member of technical staff. You know, it's about ultimately your contributions, and we see a lot of ex-managers, you know, becoming ICs and, and-- 'cause they wanna go back and build code.”

Claim: Modal treats management and individual-contributor roles as equal in prestige and emphasizes employees' contributions rather than title.

“You know, it's about ultimately your contributions, and we see a lot of ex-managers, you know, becoming ICs and, and-- 'cause they wanna go back and build code.”

Claim: Erik says some former managers have moved back into individual-contributor roles to build software directly.

“I, I still write a lot of code at Modal, and, and Akshat too, and, and pretty much everyone in, in the tech team, and a lot of people who are not in the tech team still write code.”

Claim: Erik says coding is common across Modal, including among its CEO, cofounder, technical staff, and many nontechnical employees.

“So that to me is like, you know, a, a recent phenomenon, but, but I actually don't know if it's, like, necessarily caused by GenAI. But I think GenAI has, like, greatly accelerated it 'cause it, it made it even lower. The cost of, like, writing code is even easier. So if you have, like, non-technical people, like, committing code, like, how do you, how do you review that? Like, how do you guys do code reviews?”

Claim: Erik is uncertain whether generative AI caused more employees to code, but believes it accelerated the trend by making code cheaper to produce.

“We do code reviews, but it's-- we, we think of it as optional.”

Claim: Modal performs code reviews but does not require them for every change.

“I mean, for most, uh, um, my, my view is, like, code reviews in, in large part is, like, people, in many cases, trying to just escape the, the accountability [chuckles] of, like, you know, merging things into production.”

Claim: Erik believes code reviews are often used to diffuse or avoid individual accountability for production changes.

“And, and if you have that, like, you know, if you have that culture, like, then to me, that's a sign that you need to invest way more in, you know, CD and, and CI and, and, and things like that.”

Claim: Erik argues that organizations relying on code reviews to diffuse accountability should invest more in CI and CD.

“Like, I mean, like, when I write code, like, occasionally, like, I'm, I'm asking, you know, authors, like, "Is this in line with how you think about things? I hope I'm not changing the architecture fundamentally." But we also make it clear internally that, like, code reviews are, are quite optional and, you know, if you're just changing docs, if you're just doing, you know, some basic stuff, like, don't wait for a human to look at it, especially in the era of AI. Why do you need a human to look at every piece of code? You don't have to. But that does require you to have, you know, very good controls- Yeah... you know, internally around, you know, testing things.”

Claim: Modal allows low-risk changes, including documentation edits, to proceed without human code review, particularly in the AI era.

“Why do you need a human to look at every piece of code? You don't have to. But that does require you to have, you know, very good controls- Yeah... you know, internally around, you know, testing things.”

Claim: Erik says optional human code review depends on having strong automated testing controls.

“Especially at an infra company. [chuckles] Yeah. And, and exactly. Like, you know, infra companies, like, we, we need one more nine than the average startup, right? Yeah. In terms of uptime.”

Claim: Erik believes infrastructure companies should target one additional nine of uptime compared with typical startups.

“But, like, you know, these are investments you, you just have to make. Like, you have to build systems that are very resilient to, like, you know... You have to have rate limits everywhere. You have to decouple systems. You have to make sure that, like, nothing is a single point of failure. You have to have extremely good test coverage.”

Claim: Erik says infrastructure companies should build resilient systems with rate limits, decoupling, redundancy, and very high test coverage.

“Agents are actually really good at writing tests these days, which was like, you know, a few years ago, I, I thought, like, one of the major things missing in agents. Now they're actually quite good at, like... 'Cause I think it takes a little bit of, like, sort of, kind of like need to step out of what you're trying to do for a second and be almost a little bit like devil's advocate to write good tests. But now they're actually quite good at writing tests too. So I think, I think all these things help.”

Claim: Erik says AI agents are now good at writing tests, unlike a few years earlier when he considered this a major weakness.

“Do you guys use, like, frontier models for coding, or do you guys-- have you guys experimented with, uh, like, the open source ones? We experiment with everything. But yeah, I mean, you know, initially it was a lot of Cursor, then, you know, of course Claude and Codex, and, and now a lot of people are using, you know, Nightshift and, and all these open source stuff or, or other stacks. So I don't necessarily have, like, a strong point of view. Like, I, I want people to experiment and try different things. Like, we, you know, use frontier models, use open source models.”

Claim: Modal tests frontier and open-source coding tools, and Erik prefers experimentation over committing to one coding model or stack.

“We're trying to do a little bit more dogfooding internally now that we're actually running inference for a lot of the open source models, like, at a very large scale. I think it's also helpful to, like, use our internal endpoints for a lot of agentic development. Uh, so we try to promote that a little bit. Hmm.”

Claim: Modal is expanding internal use of its large-scale open-source inference endpoints for agentic software development.

“Something I read maybe, like, I don't know, a year ago, maybe like two years ago, you were saying that you didn't think that there was gonna be, like, a big on-prem business for you guys. Um, have you maybe, like, changed your opinion on that? No, I have not changed my opinion. Um, I think Modal to me, I, I'm a cloud maximalist, and maybe to take a step back, like, when, when the cloud came in two thousand and eight, two thousand and nine or whatever, my first reaction was like, "That's crazy.”

Claim: Erik remains skeptical that Modal will build a major on-prem business and identifies as a strong advocate of cloud computing.

“Like, why would I wanna take my code and run it on someone else's computer? That's, like, insane." And then, like, six months later, I was doing it. And I think, you know, like, took me six months to change my mind.”

Claim: Erik says he initially opposed running code on someone else's computer but adopted cloud computing roughly six months later.

“It took probably enterprises, like, another fifteen to twenty years. But, like, you look at, you know, vendors like Snowflake or, I mean, AWS, and, like, you know, I, I think there's many examples of Datadog, et cetera. Like, the cloud is clearly winning, right? Like, and, and the multi-tenant systems are clearly winning. And, you know, to me, on-prem looks increasingly like sort of a legacy business model.”

Claim: Erik says enterprise cloud adoption took about 15–20 years and believes cloud and multi-tenant systems are prevailing over on-prem deployments.

“And, and I actually think, you know, BYOC, you know, sort of VPC peering, like, all of these things, you know. Like, to me, they, they reflect sort of an outdated security model of, like- We need to protect everything inside our own network, right? Like using the network as a moat. And that I don't think is true anything-- in anymore, right? Like we, we've sort of moved on to more like actually robust security paradigms like, you know, uh, zero trust and things like that. And so to me, BYOC and those things, like they, they look increasingly as like outdated, at least from a security perspective.”

Claim: Erik views BYOC and VPC peering as increasingly outdated security approaches compared with robust paradigms such as zero trust.

“From our perspective also, like the fact that we have full control over our entire infrastructure stack, we rent all the hardware, we deploy, we manage all the infrastructure, we deploy and like manage health and observe, you know, all the metrics on all the hosts. It means we can iterate very fast and build a greater product.”

Claim: Modal's control over its rented hardware and full infrastructure stack enables rapid iteration and product improvement.

“And, and I think that's very, very valuable versus, you know, customers in the past who had enterprise editions that they had to deploy on customers' environment. They had to be very, you know, they have to think very carefully about release management and, and having depen-- like, like we don't care. Like if we wanna introduce a new database, like we can introduce a new database 'cause we have control over it. Like we don't need to think about what customers have access to. So to me, multi-tenancy is, is, it has so many different benefits.”

Claim: Erik believes multi-tenancy benefits Modal by enabling centralized control and faster infrastructure changes without customer-specific release constraints.

“I mean, that being said, like you know, if we were to work with like you know, some very large Fortune five hundred customer in the future and you know, they ask for BYOC, you know, we've done this in the past, we'll give them a number. Like, you know, that number is very high and no one said yes to it so far, but like who knows? Like, you know- Yeah... at, at, at the right price, I, I, I would still build it.”

Claim: Modal is willing to build a BYOC deployment for a large customer at a high enough price, but no customer has accepted its quote to date.

“And, and I think there's like maybe separately like a, a another thing which I think is interesting, which is like in theory, BYOC might also be a way to work with and, and get capacity in certain ways. Like we, we sometimes work with Neo labs, and those Neo labs have already have very large capacity reservations. And you know, the question is like you know how, how do we, how do we get them to run modal when they already have capacity? And BYOC might be a solution. We can still keep the control plane in, in our environment.”

Claim: Erik sees BYOC as a possible way for Modal to use customers' existing capacity reservations while retaining control of the control plane.

“And the other flip side of that is like we could maybe even run like other people's stuff on that capacity. So, so there's some like kind of fun stuff you could do in theory where like you can sort of you know, like kind of aggregate all that capacity into one big pool. So, so we're, we're excited about some of those ideas.”

Claim: Erik says BYOC might allow Modal to aggregate customer-owned capacity into a shared pool and run other workloads on it.

“But in general like I feel like on-prem, I don't know if we're ever gonna do. But BYOC like maybe. So- Bring your own cloud. Yeah, yeah. I keep saying BYOC. I should explain. Yeah, it's okay. It's okay.”

Claim: Erik is uncertain that Modal will enter on-prem deployments but considers BYOC a possible offering.

“I mean, like I mentioned, like in, in this capacity situation, the, a lot of the money, a lot of the cash flow I should say that's required is, is capacity prepayments, right? And, and so that to me represents the best use of, of, of cash today. Uh, we're ramping up capacity, capacity very aggressively. So that to me is, is the, the greatest use of cash right now.”

Claim: Erik says capacity prepayments are Modal's best current use of cash as the company expands capacity aggressively.

“Maybe we move into, you know, other types of deployment models in the future, ownership or whatever, but I think there's always gonna be a cash component of that, like some sort of down payment or prepayment of some sort of that.”

Claim: Erik expects future capacity models, including ownership, to continue requiring some upfront cash payment or prepayment.

“There's of course other things like you know, salaries and office spaces and stuff like that, but that actually in the grand scheme of like our, our, uh, financials, like it's, it's basically a rounding error right now.”

Claim: Erik says salaries and office expenses are negligible relative to Modal's overall financials.

“Yeah. Interesting. Do you think that's gonna change? No. I, I think you know, the, the sort of operating leverage that we have in this business is kind of insane. Like you know, we have two hundred employees. You know, they cost a lot of money, but you know, the, the, the GPUs cost a lot more money. Yeah.”

Claim: Erik says Modal's GPU expenses exceed its payroll costs despite having around 200 employees.

“Capacity and the sort of financial complexity and you know, in general like how do we, how do we think about that space? How do we think about pricing and packaging? How do we think about the sort of tension between ownership and renting? Should we be going a more Neo cloud direction? Should we go in the opposite direction, the sort of counter position? Like that's probably the biggest thing that I think a lot about. Uh, the second biggest thing I think a lot about is, um, probably around like more like product strategy.”

Claim: Erik says Modal's primary current focus is managing capacity and its financial trade-offs, including pricing, packaging, ownership, and renting.

“Like I think a lot about like what is the next building blocks? Like if, if we, you know, do more like agentic stuff, like what are the things we need? You know, are there certain storage primitives or compute primitives or... In general like what is the thing that the world is gonna wake up and realize they need over the next few years? And, and ideally we have that at that point. We've been pretty good in general at like you know, anticipating demand and building products in advance of it with the serverless functions and the sandboxes.”

Claim: Erik says Modal's second major focus is identifying the infrastructure building blocks that future agentic applications and markets will need.

“And, and ideally we have that at that point. We've been pretty good in general at like you know, anticipating demand and building products in advance of it with the serverless functions and the sandboxes.”

Claim: Erik believes Modal has successfully anticipated demand and built products ahead of it, including Functions and Sandboxes.

“So what's the next thing? Then I would say like actually the third thing I think a lot about is like more around like culture and company building and stuff like that. I mean, we recently co- crossed two hundred people. We're, we're growing extremely fast. We'll probably hit two hundred fifty by the end of the year.”

Claim: Modal has exceeded 200 employees and is expected, with uncertainty, to reach about 250 by year-end.

“Like when you're growing very fast, you have two hundred fifty people that are all trying to figure out how they can add the most value to Modal and, you know, making sure that they have the right context, that they have the right motivation, that they're excited to come into work every day 'cause they get to build something incredibly cool and they know, you know, what makes Modal successful, what they can do, aligning the incentives and, and making sure that, you know, we, we, we-- people have like a very positive culture where they have the sort of implicit trust to do a lot of stuff, but also the accountability to like you know deliver the best things and you know, and, and be held accountable to like if they're not doing it. To me those are like you know super critical things around this scale.”

Claim: Erik says rapid company growth requires clear context, motivation, aligned incentives, employee trust, and accountability.

“Like when, when you have like ten, twenty, thirty people, you can all just like kinda sit in a room and like it's all kind of implicit and you kinda just like feel it. People can sort of read, read me. Like they can tell me when I get a little bit annoyed. They can tell when I'm happy. But like now we're two hundred fifty people and like people don't necessarily see it and you know of course there's like a lot of new people that are like maybe a little bit nervous to like talk to CEO and like all this stuff.”

Claim: Erik believes small teams can maintain culture informally, whereas companies around 250 people need more deliberate cultural communication.

“Like it, it's, it's-- there's like the Dunbar number which I think is like more sort of you know fake than, than anything, like at least like the story behind it. But like it is actually kinda real still. Like there's something around like a hundred and fifty, two hundred where like laying the foundation of the culture is very, very important. How many people are you guys trying to hire over the next year?”

Claim: Erik doubts some claims about the Dunbar number but believes companies face an important culture-building transition around 150–200 employees.

“We've grown about four X on an annualized basis in the last twelve months. I think that's like an unsustainable [laughs] growth rate on a relative basis.”

Claim: Modal's annualized growth over the past year was about 4×, which Erik considers unsustainable as a continuing growth rate.

“I've seen that, you know- You can double the team every year, and that's not super hard to do. But four X, what you start to see is, you know, cultural drift. You start to see islands of different working styles.”

Claim: Erik says 4× annual team growth can create cultural drift and fragmented working styles, whereas doubling a team annually is manageable.

“You start to see people doing, like, very speculative things. Like, that, that's something I always watch out for in the engineering team. You go and talk to the eng-- an engineer, and I'm like: "What are you working on?" And, and it turns out they're, like, building something, like, kind of like fringe.”

Claim: Erik monitors the engineering team for speculative or fringe work as the company scales.

“I, I think by the end of the year, maybe two fifty. By the end of the next year, you know. But, but, like, you know, we're gonna keep growing, like, maybe two and a half, three X.”

Claim: The speaker estimates that Modal may have around 250 employees by the end of the current year and expects the company to continue growing by roughly 2.5–3×.

“So, so it's very likely we'll end up, like, next year, like, seven hundred people, something like that. And I'm gonna assume most of those are engineers. Most, most are engineers.”

Claim: The speaker expects Modal to have approximately 700 employees next year, with most of them being engineers.

“Like, what kind of- We're ramping up sales a lot too. Okay. What kinds of engineers? It's primarily systems engineers.”

Claim: Modal is increasing its sales hiring and primarily seeks systems engineers.

“I mean, we've always been, like, very deep into sort of, you know, Linux and file systems and scalable systems and networking and, and cloud and, and stuff like that.”

Claim: Modal has historically focused on Linux, file systems, scalable systems, networking, and cloud infrastructure.

“So, so that, that's, that's the core of what we're looking for, like, people who are really good at those things. We're also looking a lot at, like, you know, more like... 'Cause we moved up in the stack a little bit.”

Claim: Modal seeks people with strong systems skills and has expanded somewhat higher in the software stack.

“Like, we're doing a lot of work in the LM inference space.”

Claim: Modal is doing substantial work on language-model inference.

“So now we're dealing with a separate set of concerns around, you know, CUDA and, like, GPUs and, and, and, and MFU, you know, like maxing out the tokens that you can get per GPU and, and, and a lot of that things.”

Claim: Language-model inference work involves CUDA and GPU concerns, including maximizing the number of tokens produced per GPU.

“And, you know, e-even though I actually, like, will say a lot of the work [chuckles] in that space, in my opinion, turns out is, like, more systems driven still, 'cause it turns out, like, KV caching and routing is actually, like, is still, like, very, like, systems-like.”

Claim: The speaker believes much language-model inference work remains systems-driven, since KV caching and routing are systems-like problems.

“Like, like, these are very, like, systems heavy pr-problems. But yeah, a little bit more, like, sort of GPU performance, that kind of stuff. We're starting to build up a research team.”

Claim: The speaker characterizes the relevant problems as systems-heavy and says Modal is also increasing its GPU-performance work and building a research team.

“We're very excited about, you know, g-moving into stuff around, like, you know, post-training and, and distillation and, and better methods for, for quantization and things like that.”

Claim: Modal is investing in work on post-training, distillation, and better quantization methods.

“For your customers or for internally? Both. I, I think it's, like, a little bit like with the FDEs. Like, I, I, I think we initially will do a lot of stuff with our customers to optimize their models, but over time figure out how do we turn those into products. I think we'll sort of arguably fail in a certain way if every time a customer needs to optimize a model, we have to go and work with them and, you know.”

Claim: The speaker believes Modal would be failing in an important respect if each customer model optimization continued to require direct, bespoke work with the customer.

“For your customers or for internally? Both. I, I think it's, like, a little bit like with the FDEs. Like, I, I, I think we initially will do a lot of stuff with our customers to optimize their models, but over time figure out how do we turn those into products. I think we'll sort of arguably fail in a certain way if every time a customer needs to optimize a model, we have to go and work with them and, you know.”

Claim: Modal plans to initially help customers optimize models and eventually productize the resulting work.

“Long term, it has to be turned into products. And, and so with the FDEs and with the research team, initially, you're gonna have to do a lot of work with the customers, and we are that.”

Claim: Modal intends for its customer-facing work, including work by FDEs and the research team, to become products over the long term.

“But over time, we're also building a lot of internal tooling and a lot of overtime internal products that are fully self-serve.”

Claim: Modal is developing internal tools and products intended to become fully self-serve over time.

“Uh, so we're working on a lot of stuff right now in, in the post-training space, but also in, in the GPU performance space. So a lot of cool stuff. A lot of cool stuff. Back to the product strategy stuff that you were talking about. Yeah. Um, yeah, like, what are your thoughts? Like, what do we need over the next year, two years? I think it's incredibly hard to say.”

Claim: The speaker believes it is extremely difficult to predict what Modal will need to build over the next one to two years.

“And, and i-if I said, like, I know exactly what we need to build, like, I think I would be, like, delusional. [chuckles] I think what, what I've...”

Claim: The speaker believes it would be delusional to claim certainty about exactly what Modal needs to build.

“what we've seen so far, you know, and, and what I kind of believe in on a more general level is, like, we need better building blocks.”

Claim: The speaker believes the industry needs better software and infrastructure building blocks for the new era of applications.

“Like, this is a new era of, of software, and we're unlocking a whole new type of applications that could not be built a few years ago.”

Claim: The speaker believes the current era of software is enabling new kinds of applications that were not buildable a few years earlier.

“You know, doing, you know, all the GenAI stuff, code that, you know, LMs that output code that, you know, it needs to run, like, in an isolated way.”

Claim: Some generative-AI applications use language models to generate code that must execute in an isolated environment.

“Like, there's so many, like, new primitives that are needed to, to work with GPUs and work with LMs.”

Claim: The speaker says that working with GPUs and language models requires many new software primitives.

“And, you know, we built some building blocks so far and, and, you know, that's unlocked, you know, a bunch of applications that, that the world, you know, has seen over the last few years. Not just of us, of course.”

Claim: The speaker says Modal has built infrastructure building blocks that contributed to applications emerging over the past few years, while acknowledging others contributed too.

“But I think over the next few years, like, there's gonna be so many more cool things. And so I, I, I mentioned some stuff. Like, I'm very bullish on training. I'm very bullish on batch jobs.”

Claim: The speaker expects many new applications over the next few years and is especially optimistic about AI training and batch jobs.

“I'm very bullish on, you know, certain things around, like, agents and, and low latency and, and things like that. But, but, like, in the end, we consider ourselves, like, an infrastructure shop, right? Like, we're not trying to build, you know, very narrow things.”

Claim: Modal sees itself as an infrastructure company rather than a builder of narrowly focused products.

“I'm very bullish on, you know, certain things around, like, agents and, and low latency and, and things like that. But, but, like, in the end, we consider ourselves, like, an infrastructure shop, right?”

Claim: The speaker is optimistic about certain AI-agent applications and low-latency use cases.

“Like, we try to build, like, very general purpose building blocks and put them in front of customers, and then those customers will combine those building blocks and build some incredible applications that we could have never anticipated.”

Claim: Modal aims to provide general-purpose building blocks that customers can combine to create applications the company could not have anticipated.

“I'm fully con-- I, I think s- I think the median sandbox in Modal today is, like, eight seconds long or something like that. Like, which is kind of incredible when you think about it.”

Claim: The speaker estimates that the median sandbox runtime on Modal is approximately eight seconds at present.

“And, and over time, I think it might get even shorter. We wanna support long-running sandboxes too. Like, I think there are some, like, very valid s- use cases around long-running sandboxes.”

Claim: The speaker expects sandbox runtimes may shorten over time, while Modal also intends to support long-running sandboxes because they serve valid use cases.

“But, like, my vision is, like, you know, the world needs also... We need to figure out this. Like, how, how can we just massively parallelize things and reducing the overhead of starting and stopping sandboxes?”

Claim: The speaker wants systems to massively parallelize workloads while reducing sandbox startup and shutdown overhead.

“I don't know if I ran an IPO-ready startup [chuckles] engaged in CapEx arms race with a competitor. It would be nice to coordinate and pace the arms race a bit. No comment.”

Claim: The speaker says coordination and pacing would be desirable in a capital-expenditure arms race with a competitor, but declines to elaborate.

“[laughs] "The biggest downside of GPU scarcity is not that it's hard to get GPUs, but that almost all capacity is sold as three years, as a three-year rectangle starting three to six months in the future.”

Claim: The speaker argues that GPU scarcity is especially problematic because most capacity is sold in three-year contracts starting three to six months later, unlike the demand patterns of most users.

“Like, one thing I think Modal has always been very, very good at is fully on-demand capacity, right? Like, if a customer comes to us and need a hundred GPUs, like, we can typically give them that, like, immediately.”

Claim: The speaker says Modal can typically provide 100 GPUs immediately when a customer requests them, as part of its on-demand capacity offering.

“You don't even have to talk to a salesperson. And a lot of workloads in, in, in real life, like, they don't look like these, like, flat, you know, rectangular workloads.”

Claim: The speaker says customers can access Modal capacity without speaking to sales, while real-world compute workloads often vary rather than remaining flat and rectangular.

“Like, they scale up and down throughout the twenty-four hour cycle. They maybe grow over time. They maybe have certain burst periods. And to me, that is a challenge with a lot of customers today. Like, the...”

Claim: The speaker says compute workloads may fluctuate over the day, grow over time, or experience bursts in demand.

“And, and I do think it's a legit problem for, for the space. It's like the only... The prime way to get capacity is to, to buy these rectangular blocks. It's not great. I think the world needs to vol...”

Claim: The speaker believes it is a genuine industry problem that compute capacity is primarily available through fixed, rectangular blocks.

“And, and that's fundamentally, I mean, I'm, I'm very biased, but, like, that's why I think Modal Can play a very large role in, in that sort of, you know, whatever you wanna call it, sort of duration transformation of offering more flexible capacity.”

Claim: The speaker believes, while acknowledging his bias, that Modal can play a major role in making compute capacity more flexible in duration and availability.

“And we can do that be-we-- because we can aggregate many, many thousands of customers, and the demand gets very smooth in aggregate. So that's a lot of the value prop of Modal is exactly that.”

Claim: The speaker says Modal can provide more flexible compute capacity by aggregating thousands of customers, smoothing demand across them.

“Would you say that's a differentiator? Like the- I think so. Oh, really? Okay. Yeah, massive differentiator. A-actually, in fact, like, I think Modal's like...”

Claim: The speaker considers Modal’s flexible capacity offering a major competitive differentiator.

“If you wanna be, like very sort of reductive, like you can break down Modal's value prop into two things. One is the developer experience, the ability to like iterate and like build things and not have to think about infrastructure. The other thing is the capacity stuff. Yeah.”

Claim: The speaker identifies Modal’s two main value propositions as developer experience that abstracts infrastructure and access to compute capacity.

“The more and more I think about Modal and, like where we're going in, in the long run is like we're building, like we're aggregating the world's compute capacity, and we're very good at slicing it out, like very granularly 'cause we've built all this work around, you know, scaling up and down extremely fast.”

Claim: The speaker describes Modal’s long-term strategy as aggregating compute capacity globally and allocating it in fine-grained units by scaling workloads up and down quickly.

“You know, all this stuff around GPU snapshotting. We're working some, some ways to make it much, much faster. And that model works at very large scale, like really, really well 'cause you can aggregate lots of customers.”

Claim: The speaker says Modal is improving GPU snapshotting speed and that its customer-aggregation model works well at large scale.

“Every month I keep saying our sandbox revenue might apply to it, but it keeps going up thirty percent every month. [chuckles] It's kinda nuts.”

Claim: The speaker says Modal’s sandbox revenue has increased by 30% each month.

“Like I don't under-- you know, like every month I'm like: Well, you know, sandboxes grew a lot, but like clearly they kinda plateaued, and then they keep growing even more. So, so I, I, I don't know.”

Claim: The speaker says he repeatedly thought Modal’s sandbox growth had plateaued, but it continued to grow.

“And a lot of VCs, like, they're like: "How did you know sandboxes was gonna be big?" Like, and, and the truth is like, I didn't. Like, [chuckles] like- [chuckles]...”

Claim: The speaker says he did not know in advance that sandboxes would become a major part of Modal’s business.

“I, I thought it was gonna be bigger than it was in twenty twenty-three for sure, but like the fact that sandboxes is a third of our revenue today, it's, it's insane and, you know, we're obviously very happy about it and we, we have like a very large team investing in sandboxes.”

Claim: The speaker says he expected sandboxes to be more significant in 2023 than they were, and that they now make up one-third of Modal’s revenue.

“I, I thought it was gonna be bigger than it was in twenty twenty-three for sure, but like the fact that sandboxes is a third of our revenue today, it's, it's insane and, you know, we're obviously very happy about it and we, we have like a very large team investing in sandboxes.”

Claim: Modal has assigned a large team to work on sandboxes.

“You know, I don't wanna like trivialize it. Like we obviously... I do wanna s- give ourselves like a lot of credit for, for seeing sandboxes for what it is today a little bit back in even twenty twenty-three.”

Claim: The speaker believes Modal deserves some credit for recognizing the potential of sandboxes in 2023.

“But I, I, I think, you know, if, uh, like I probably would've never, you know, taken a bet that revenue would be this large from sandboxes and the market demand would be so much.”

Claim: The speaker says he probably would not have predicted that sandbox revenue and market demand would become as large as they have.

“There is an interesting structural reason why infr-inference providers should exist separately from model builders. The best model keep changing a lot.”

Claim: The speaker argues that inference providers should be separate from model builders because the leading models change frequently.

“Inference providers keep-- can easily reallocate resources between models since overall demand is quite smooth.”

Claim: The speaker says inference providers can shift resources among models relatively easily because aggregate demand is fairly smooth.

“So I mean, if you think about all the Chinese labs and, you know, in the future, hopefully more US labs too, every time a new lab releases, you know, the latest and greatest open source models, like you see big shifts in demand, you know, for, for these models.”

Claim: The speaker says new open-source model releases from Chinese labs, and potentially more U.S. labs in the future, can cause substantial shifts in demand among models.

“And, and so I think a challenge with, you know, i-if all these labs independently like held capacity, they would see when they launch a new model, demand would go up a lot.”

Claim: The speaker says that if model labs independently held compute capacity, demand for a lab’s capacity could rise sharply when it launches a new model.

“And then another lab launches a better model, and then the demand goes down. Yeah. You know, but, but of course, like in the background they're working on the next version.”

Claim: The speaker says demand for a model can decline after another lab releases a better model, even as the original lab works on its next version.

“You, you're seeing demand kinda shifting to like the latest and greatest Chinese lab mo-- I mean, not like-- It's not like, you know, it's just like everything just moves overnight 'cause, 'cause a lot of models and prompts and stuff like, like a lot of things like get sort of... There, there's like a certain stickiness to it.”

Claim: The speaker says demand does not move entirely overnight to newer models because existing models, prompts, and related workflows create some stickiness.

“But I, I do think, you know, there's an argument that it's very easy for Modal to like, you know, right today we're running, you know, X number of thousand GPUs for one model. Tomorrow we need to shift like, you know, fifty percent of that to another model. Sure. That's just like, that's what the system is built to handle. Yeah. No one even needs to tell us to do that.”

Claim: The speaker says Modal’s system can automatically reallocate thousands of GPUs between models, including shifting half the capacity by the next day, without a customer instructing it.

“It just magically does that. So you agree that basically the LLMs are commodities? And if you agree, then where do you think the business for the big labs is going to be? I'm not sure if that's necessarily true, right?”

Claim: The speaker does not accept as certain that language models are commodities.

“So like as long as the, the labs like repeatedly can launch new models and as long as the margin that they have on those models are such that they, they can sort of pay off, amortize like the, the cost of training those models, it could still be like a decent business, I think.”

Claim: The speaker thinks model labs could remain viable businesses if they repeatedly release new models with margins sufficient to amortize the cost of training them.

“So you don't think that the intelligence is like plateauing? No, I don't, I don't think so. I mean, I, I think it remains to be seen, right?”

Claim: The speaker does not believe model intelligence has plateaued, while acknowledging that the issue remains uncertain.

“Like models are today are like very good at like doing things that can be trained very efficiently, you know, using reinforcement learning.”

Claim: The speaker says current models perform very well on tasks that can be trained efficiently with reinforcement learning.

“So, so we've seen that initially with, you know, coding and writing simple snippets. Over time it's gotten really good at also writing production li-level code. It's gotten really good at math.”

Claim: The speaker says models have improved from generating simple code snippets to producing production-level code and performing well at mathematics.

“It's gotten really good at cybersecurity. Like these are all like very quite verifiable things. I, I still think it remains to be seen, you know, are models gonna be really good at whatever, cooking food or... I, I, I don't know. Like things that have like more like diffuse targets and maybe longer feedback loops or whatever. I think humans probably like...”

Claim: The speaker says it remains uncertain whether models will excel at tasks such as cooking that have diffuse objectives or long feedback loops.

“It's gotten really good at cybersecurity. Like these are all like very quite verifiable things. I, I still think it remains to be seen, you know, are models gonna be really good at whatever, cooking food or...”

Claim: The speaker says models have become very good at cybersecurity, a task area he considers relatively verifiable.

“I, I, I don't know. Like things that have like more like diffuse targets and maybe longer feedback loops or whatever. I think humans probably like... My feeling is like we'll be quite good and quite competitive with, with models for a long time in those domains. But you know, certainly, you know, AI models, they're, they're getting very, very good- Cool...”

Claim: The speaker expects humans to remain capable and competitive with AI models for a long time in domains with diffuse objectives or long feedback loops.

“as, as I'm sure everyone can agree with. Yeah. So obviously like the models are getting really good at coding. They're already really good at coding.”

Claim: The speaker says AI models are already very capable at coding and continue to improve.

“But I think anything that has like kind of a verifiable- A computational bio, I think you mentioned- Yeah, computational bio. I, I think those are like not exactly verifiable, but there's like certain ways you can sort of run simul-- like you can use an LLM to output predictions. Then you can do like computational like, like methods to like simulate the molecular dynamics and get like kind of feedback loops.”

Claim: The speaker sees computational biology as a possible application for language models because their predictions can be assessed using simulations and feedback loops.

“I, I think those are like not exactly verifiable, but there's like certain ways you can sort of run simul-- like you can use an LLM to output predictions. Then you can do like computational like, like methods to like simulate the molecular dynamics and get like kind of feedback loops. I mean, like I think like material science is like maybe another area where like you have a little bit of that loop where you can sort of, you know, synthetically manufacture certain materials and measure the properties, and you just kind of put an agent in the loop and like, you know, do things.”

Claim: The speaker sees materials science as a possible AI application because materials can be synthesized and measured, allowing an agent to operate within a feedback loop.

“So, so you know, I have a physics background. I'm pretty excited about that. So drugs, materials. What else? I'm drawing a blank, but I'm sure there's many other things. Cool.”

Claim: The speaker says he has a physics background and is excited about AI applications in drug discovery and materials.