Why smarter AI models could drive up compute prices 10x

“Anthropic's revenue has 10x'd year over year.”

Claim: Anthropic's revenue has grown 10x year over year for three consecutive years.

“And it's likely to do so again this year.”

Claim: Anthropic is likely to grow revenue 10x year over year again this year.

“So they ended last year with $9 billion in revenue.”

Claim: Anthropic's revenue was $9 billion at the end of last year.

“I think they'll probably end this year with somewhere between $100 billion to $150 billion in revenue.”

Claim: The speaker thinks Anthropic will probably generate $100 billion to $150 billion in revenue by the end of this year.

“Now for this trend to continue, Anthropic would need to make $1 trillion in revenue by the end of next year.”

Claim: If Anthropic's 10x year-over-year revenue growth trend continues, it would need to make $1 trillion by the end of next year.

“Of course there's no deep reason why this has to be true.”

Claim: The speaker says there is no deep reason that Anthropic's rapid revenue-growth trend must continue.

“It's a very wild conclusion and it's ultimately a question of trend NBA capabilities. Does AI get that useful by the end of next year?”

Claim: The speaker frames continuation of the revenue-growth trend as depending on whether AI capabilities become sufficiently useful by the end of next year.

“Now, the other big trend in AI is that lab compute only 3x's year over year.”

Claim: AI lab compute has been increasing by only 3x year over year.

“For a lab to keep 10x'ing revenue year over year while compute only 3x's, one of the following three things needs to happen or some combination of the three needs to happen. lab margins have to increase. Two, the price of compute has to increase. Or three, the percentage of compute that labs spend on inference rather than training has to increase.”

Claim: If an AI lab's revenue continues growing 10x year over year while its compute grows 3x, its margins must increase, compute prices must rise, the share of compute used for inference rather than training must increase, or some combination of these must occur.

“My understanding is that basically all three of these things are already happening.”

Claim: The speaker understands that AI lab margins, compute prices, and the share of compute allocated to inference are all already increasing.

“With regards to the margins, Anthropics' inference margins reportedly went from 40% to the middle of last year to upwards of 80% now favorable.”

Claim: Reportedly, Anthropic's inference margins increased from 40% in the middle of last year to over 80% now.

“With regards to compute, the spot prices for compute are more than 40% higher than they were in the February. February trough that we had earlier this year.”

Claim: Compute spot prices are over 40% higher than at the February trough earlier this year.

“And with regards to the share compute that goes to trading versus inference, in 2024, according to Epoch, OpenAI was spending just a quarter of its compute on inference, and that number is likely closer to 50%, if not higher now.”

Claim: According to Epoch, OpenAI allocated about 25% of its compute to inference in 2024; the speaker estimates the share is now likely around 50% or higher.

“Now, labs would prefer not to do this final thing, of increasing the share of compute they spend on inference.”

Claim: AI labs would prefer not to allocate a larger share of their compute to inference.

“The way the labs see the world, the whole point of inference revenue is to help convince investors to give you more money. in order to train the next bigger better model and if you're spending most of your compute on inference and you're basically declaring that AI progress has stalled and you're just now in the business of being a cloud provider now this is a less compelling business than building AGI and so the labs do not want to be in this business nor do they think they're in this world they think that within a year they'll have built models that make the current ones look extremely shitty but they need to invest”

Claim: AI labs see inference revenue as a means of persuading investors to fund training of the next, larger and better model.

“in order to train the next bigger better model and if you're spending most of your compute on inference and you're basically declaring that AI progress has stalled and you're just now in the business of being a cloud provider now this is a less compelling business than building AGI and so the labs do not want to be in this business nor do they think they're in this world they think that within a year they'll have built models that make the current ones look extremely shitty but they need to invest”

Claim: AI labs do not want to be cloud providers rather than build AGI, which they regard as a less compelling business.

“in order to train the next bigger better model and if you're spending most of your compute on inference and you're basically declaring that AI progress has stalled and you're just now in the business of being a cloud provider now this is a less compelling business than building AGI and so the labs do not want to be in this business nor do they think they're in this world they think that within a year they'll have built models that make the current ones look extremely shitty but they need to invest”

Claim: AI labs believe they will build models within a year that make today's models look extremely poor.

“a lot of their compute, the majority of their compute into doing the training and experiments that are necessary to build the next model.”

Claim: AI labs need to spend the majority of their compute on training and experiments for the next model.

“So that leaves only two options for how you can get out of this gap between the fact that lab compute only increases 3X year over year, but revenue increases 10X. Either the lab's margins have to increase so that they get this surplus or the price of compute has to increase so that everybody in the stack below the lab gets a surplus.”

Claim: If lab compute grows 3x year over year while revenue grows 10x, either lab margins must increase to capture the surplus or compute prices must rise so suppliers below the labs capture it.

“It's not clear to me which world we end up in.”

Claim: The speaker is uncertain whether AI lab margins or compute prices will rise to resolve the gap between revenue and compute growth.

“Do we end up in a world where we go from 80% margins for some of the top models to greater than 90% margins if the lab margin effect dominates? Well, that would require the leading model to be so far ahead of the competition because the nature of margins,”

Claim: If higher lab margins are the main outcome, margins on some top AI models could rise from 80% to over 90%, requiring the leading model to be far better than competitors.

“why they exist in a market economy, is that the thing you are serving is so much better than what somebody else could go get and replace you on the market. But it's just really wild for me to consider that.”

Claim: The speaker says high margins in a market economy arise when a product is substantially better than available substitutes.

“that the margins for something like intelligence will be greater than 90% and they don't get competed away at that level.”

Claim: The speaker finds it surprising that intelligence could sustain margins above 90% without competition eroding them.

“So that leaves only one other possibility of this escape valve between these two trends, which is that the price of compute has to increase.”

Claim: The speaker identifies rising compute prices as the remaining way to reconcile faster lab revenue growth with slower compute growth if lab margins do not absorb the difference.

“As I mentioned, this is already starting to happen.”

Claim: Compute prices have already begun to rise.

“And the effect is even stronger when you look at the tranche of compute that the frontier labs actually need to accumulate because they can't just go out and buy a spot instance.”

Claim: Compute price increases are stronger for the large-scale capacity frontier labs need to accumulate, since they cannot rely on buying spot instances.

“They need to make sure that they get enough scale to get... at really good efficiency and flexibility and also that they have the kind of compute that lends itself to the security they need for their own weights and for their customers information so i think a relevant case study here is to look at the computes that google and anthropic are renting from spacex google for example is paying 900 million dollars a month for 110 000 gpus that are a blend of gb200s and gb300s the price that google is paying here”

Claim: Frontier labs need large-scale compute that provides efficiency and flexibility and protects their model weights and customers' information.

“at really good efficiency and flexibility and also that they have the kind of compute that lends itself to the security they need for their own weights and for their customers information so i think a relevant case study here is to look at the computes that google and anthropic are renting from spacex google for example is paying 900 million dollars a month for 110 000 gpus that are a blend of gb200s and gb300s the price that google is paying here”

Claim: Google pays $900 million monthly to rent 110,000 GPUs, a mix of GB200 and GB300 units, from SpaceX.

“is 2x the spot price per hour for those GPUs.”

Claim: Google pays twice the hourly spot price for the GPUs it rents from SpaceX.

“And that spot price itself is more than 40% higher than it would have been in February.”

Claim: The GPUs' spot price is over 40% higher than it was in February.

“I want to emphasize the key conclusion here, that as AI models get smarter, they'll be better able to monetize the same amount of compute.”

Claim: Smarter AI models will be able to generate more revenue from the same amount of compute.

“If a true human level software engineer could run on an H100 equivalent, then at today's prices for software engineers, that H100 should rent for over $250K a year.”

Claim: If an H100-equivalent could run a human-level software engineer, its annual rental value at current software-engineer prices would exceed $250,000.

“that H100 should rent for over $250K a year. That's over 15x the current spot price for an H100.”

Claim: The hypothetical annual rental value of an H100 running a human-level software engineer would exceed 15 times its current spot price.

“That's over 15x the current spot price for an H100. And this is not even accounting for the fact that your AI can work nights and weekends.”

Claim: The H100 rental-value calculation does not account for AI being able to work nights and weekends.

“Of course, you might expect that if we had 10 million extra software engineers suddenly appear in the economy, the marginal value of a software engineer would decrease and thus the revenue that that H100 would be able to generate would not be 15x higher than it is right now.”

Claim: The speaker suggests that a sudden addition of 10 million software engineers might lower the marginal value of software-engineering labor and thus reduce the revenue an H100 could generate below 15 times its current level.

“But I actually don't know if this is true.”

Claim: The speaker is uncertain whether a sudden increase in software-engineer supply would reduce the marginal value of software engineers and H100-generated revenue as described.

“then this would be the classic lump of labor fallacy. For example, economists generally believe that high school immigration does not decrease wages in the long run because of how innovation and specialization increase the value of labor.”

Claim: The speaker says economists generally believe high immigration does not lower long-run wages because innovation and specialization raise the value of labor.

“Maybe this labor supply shock will be so big and so fast that we can't count on this general heuristic anymore.”

Claim: The speaker thinks a labor-supply shock from AI might be so large and rapid that the usual heuristic about labor supply and wages cannot be relied on.

“But if you believe what standard economics says, then the marginal value of labor and thus the marginal value of compute. Feud should stay astonishingly high.”

Claim: If standard economics is correct, the marginal value of labor and consequently compute should remain extremely high.

“One of the things that would happen is that as the top labs get better and better at monetizing compute and the cost of compute increases, it becomes harder for anybody else to compete against them because they have to bid for this resource against somebody who is basically able to make better use of it.”

Claim: If leading AI labs improve their monetization of compute while compute prices rise, other competitors will find it harder to compete for compute against labs that use it more productively.

“Another thing that will happen, and I think this is actually the most interesting implication of this whole thought exercise, is that if you can train the best, most efficient model. then you'll be able to charge much higher margins than you can today.”

Claim: A lab that trains the best and most efficient model will be able to charge higher margins than it does today.

“This is the Alkin-Allen effect in economics, and what it's basically saying is that if it costs $20 an hour to rent an H100, then it would be extremely stupid to use a weaker, less efficient model because it's going to burn more tokens on your expensive compute to get the exact same result.”

Claim: When H100 compute costs $20 per hour, a weaker, less efficient model wastes money by using more tokens to achieve the same result.

“So labs will be able to charge a much larger premium if they can train a model that better economizes this scarce input.”

Claim: AI labs can charge a larger premium for models that use scarce compute more efficiently.

“if you have a model that can get the same result by using less compute, then you've in some sense created more compute and the value of compute is going to increase.”

Claim: A model that produces the same result with less compute effectively expands available compute and increases compute's value.

“Another thing that will happen is that a lot of current popular applications of AI will probably get priced out.”

Claim: The speaker expects that many popular AI applications will probably become uneconomical as compute prices rise.

“The reason AI is relatively cheap right now is that AI just can't do a lot of things that top humans can do.”

Claim: AI is relatively inexpensive today because it cannot perform many tasks that top human workers can perform.

“But this at some point will no longer be the case.”

Claim: The speaker expects AI eventually to become capable of tasks that top human workers can perform.

“And at that point, Google or Anthropic or... OpenAI will be willing to pay more for the tokens to automate AI research than you or I will be willing to pay to make more AI slop talk.”

Claim: The speaker expects Google, Anthropic, and OpenAI to pay more for AI tokens used to automate AI research than consumers will pay for low-value AI-generated content.

“I'm a bit worried that this kind of analysis honestly pattern matches a lot onto the ways that people in the past have been wrong about scarcity.”

Claim: The speaker worries that the argument about compute scarcity may repeat historical mistakes in predicting scarcity.

“I'm, for example, thinking of the famous Simon-Erlich bet. Paul Erlich was this famous doomer about population growth, and he made this bet that a basket of commodities would increase in price rather than decrease in the decade preceding 1990.”

Claim: Paul Ehrlich bet that a basket of commodities would increase in price during the decade preceding 1990, amid his concerns about population growth.

“And this is a very famous bet because it's supposed to illustrate how how Ehrlich's Malthusian worldview was wrong, and how he did not anticipate the way in which market signals and human ingenuity can find better ways to economize scarce inputs.”

Claim: The speaker says the Simon-Ehrlich bet is commonly presented as showing that Ehrlich's Malthusian worldview was wrong because he underestimated how markets and human ingenuity can conserve scarce resources.

“I'm guessing that the analogy to this bet is probably wrong.”

Claim: The speaker guesses that comparing compute scarcity to the Simon-Ehrlich bet is probably a bad analogy.

“Other analysis has shown that if that bet had been made in a different decade, Erlich might well have won.”

Claim: The speaker says other analyses suggest Ehrlich might have won the commodity-price bet had it been made in another decade.

“I think the supply of compute is much less elastic and much less capable of absorbing large demand shocks and much less capable of being accommodated by using different substitutes than the extraction of different metals is.”

Claim: The speaker believes compute supply is less elastic, less able to absorb demand shocks, and less substitutable than metal supply.

“To illustrate why I think this 3x in compute capacity year over year is hard to budge or potentially even... actually even sustain is that I don't see how any of the three elements that constitute that 3x can be much accelerated.”

Claim: The speaker doubts that the three sources of 3x annual compute growth can be accelerated substantially or even sustained.

“So 1.4x of that is coming from Moore's law.”

Claim: The speaker attributes approximately a 1.4x increase in annual compute capacity to Moore's law.

“Far from increasing it, I think it'll be a miracle if you can just keep it going for a few more years.”

Claim: The speaker thinks it would be remarkable if Moore's law continued for even a few more years.

“1.2x is coming from building new fabs. This process is ultimately going to be bottlenecked up to 2030 and potentially even beyond by just building new ASML UV machines. gains.”

Claim: The speaker attributes about 1.2x annual compute growth to new fabs and says fab expansion will be bottlenecked through 2030 or later by the construction of new ASML UV machines.

“And 1.8x comes from the fact that AI is absorbing a lot of wafer allocation that was previously going to smartphones and PCs.”

Claim: The speaker attributes about 1.8x annual compute growth to AI absorbing wafer capacity previously allocated to smartphones and PCs.

“This is probably going to hit a wall by the end of next year when at the leading edge N3 nodes at TSMC, AI will have gone from 60% to 86%. At some point, you have just absorbed all leading edge wafer capacity for AI.”

Claim: The speaker expects AI's share of TSMC's leading-edge N3 wafer capacity to rise from 60% to 86% and probably hit a limit by the end of next year.

“AI will have gone from 60% to 86%. At some point, you have just absorbed all leading edge wafer capacity for AI. AI and you can't keep increasing this number.”

Claim: The speaker says AI's share of leading-edge wafer capacity cannot continue rising once it has absorbed all available capacity.

“So I don't know how we get even to continue to do 3x compute scaling year over year for the next few years, much less go beyond that.”

Claim: The speaker is uncertain how AI compute can sustain 3x year-over-year growth for the next few years and doubts it can grow faster.

“Now, I want to clarify that at some point in the future, compute will get cheap again.”

Claim: The speaker expects compute to become inexpensive again at some point in the future.

“At some point, we'll just have robots that can convert shores of silica sand and mines of copper into new computer chips.”

Claim: The speaker expects robots eventually to turn silica sand and copper ore into computer chips.

“And then the price of computers is basically the raw inputs and the tools required to do this processing.”

Claim: The speaker expects computer prices eventually to be determined mainly by raw materials and the tools used to process them.

“I'm just talking about this current pre-singularity regime where AI compute merely three X's year over year, which is not enough to offset how much more valuable. AI is becoming over time.”

Claim: In the current pre-singularity period, AI compute growth of 3x per year is insufficient to offset the increasing value of AI.

“By the way, the fact that anthropic revenue has been 10x in year over year, whereas their compute has only been 3x in year over year, I think illustrates how strong the economies of scale are in the model business.”

Claim: The speaker says Anthropic's 10x annual revenue growth alongside 3x compute growth illustrates strong economies of scale in the AI model business.

“When you train a model, you just have to spend this one-time cost to learn all these different skills that then get to be shared across all your users.”

Claim: A model's training cost is incurred once, while the skills learned can be shared across all users.

“This is very unlike human labor where each instance has to be retrained from scratch.”

Claim: Unlike model training, each instance of human labor must be retrained from scratch.

“I wish we didn't live in a world with such strong economies of scale for intelligence because I'm worried about power concentration, but it seems we do.”

Claim: The speaker is concerned that strong economies of scale in intelligence could concentrate power.