How Anthropic ships the #1 LLM in the World safely: Separating the Brain from the Hands

“So, uh, trillion 12 month revenue per employee is now six point six million, uh, as compared to one year ago it was one point four million. So they've grown almost five times the, uh, revenue per, per employee.”

Claim: Anthropic's 12-month revenue per employee increased from $1.4 million to $6.6 million over one year, nearly fivefold.

“Um, this is obviously tracked by seeing a significant increase in revenue per dollar of compute.”

Claim: Anthropic's revenue per dollar of compute increased significantly.

“And it's notable that basically in Q2 of twenty twenty-five, Anthropic was basically losing half a dollar for every dollar of revenue, and now it's making two dollars for every one dollar in rev-- in cost.”

Claim: Anthropic's economics shifted from losing about $0.50 per revenue dollar in Q2 2025 to making $2 per cost dollar at the time of the conversation.

“Um, I-- the one consequence of this entire increase is that Anthropic has just reported its first profitable quarter, like, not adjusted profit, which is profit after we take out all the things that cost us money. No, this is generally accepted accounting principles GAAP profit, uh, of nineteen percent, and that is enormous, again, completely flipping from one year ago. Um, more than flipping.”

Claim: Anthropic reported its first profitable quarter, recording GAAP profit of 19%.

“Like, one year ago, it was losing a hundred and twenty-nine percent, uh, of every revenue, uh, dollar they came in.”

Claim: One year before the reported profitable quarter, Anthropic was losing 129% of each revenue dollar it received.

“The first is the improvement loop. Basically, there is the cycle of you have Claude. Claude leads to better models. Mo-- better models lead to more use cases. More use cases generate more feedback, and that leads to a better Claude. And then there's also the investment loop. So revenue from, uh, expanding adoption leads to higher growth.”

Claim: Anthropic describes an improvement loop in which Claude enables better models, better models create more use cases, and those use cases generate feedback that improves Claude.

“More use cases generate more feedback, and that leads to a better Claude. And then there's also the investment loop. So revenue from, uh, expanding adoption leads to higher growth. Higher growth leads to more investment, and more investment leads to more capabilities. That leads to a better Claude.”

Claim: Anthropic describes an investment loop in which expanding adoption increases growth, growth enables more investment, investment builds capabilities, and those capabilities improve Claude.

“So these two loops basically spin together and form an infinite loop that is driving the increase in efficiency until the end of time, according to the Anthropic S1. So I think that is our transition from the overall efficiency story to how they execute it.”

Claim: Anthropic's S-1 characterizes the combined improvement and investment loops as driving efficiency growth indefinitely.

“They provide customers with direct access to the primitives to build, deploy, scale their own applications that leverage Claude's capabilities within their own products and workflows.”

Claim: The Claude platform provides customers with primitives to build, deploy, and scale applications that use Claude's capabilities in their products and workflows.

“So, um, something that's also interesting in the Claude platform that, uh, was disclosed is over fifty percent of their customers with a hundred thousand more-- dollar or more revenue on a trailing 12 month basis use Claude Platform.”

Claim: More than half of Anthropic's customers with at least $100,000 in trailing-12-month revenue use Claude Platform.

“And so that is an increase in an adoption and emergence of Anthropic not only as a model lab, but as al-also a hyperscaler and a cloud platform, which I think maybe is in the name inherent in a Claude platform, in a cloud platform, but is going from theory to actual practice, and this is something that we're focusing on in the podcast.”

Claim: Anthropic is increasingly functioning as both a model lab and a hyperscaler and cloud platform.

“The other thing that I also really try to emphasize, which I talked about before, is Claude Code, Claude Cowork, they all use the same primitives that Claude Platform, um, exposes, so I think that's really interesting.”

Claim: Claude Code and Claude Cowork use the same underlying primitives that Claude Platform exposes.

“Ultimately, it looks like Anthropic is really positioning their IPO to talk about going from chatbots to copilots to virtual collaborators.”

Claim: Anthropic's IPO positioning describes a shift from chatbots to copilots and then to virtual collaborators.

“They use this term instead of agents, maybe it's more friendly, but it's a lot of the same things that you look for from agents, from long-running autonomous work to judgment, self-correction, and iteration and tool access.”

Claim: Anthropic's term “virtual collaborators” refers to agent-like systems with long-running autonomous work, judgment, self-correction, iteration, and tool access; the speaker speculates the term may sound friendlier than “agents.”

“This was recorded when Opus 5.5 was launched, and I would say overwhelmingly among all the high-taste testers that we talked to, Opus 5.5 is the best model in the world right now.”

Claim: At the time of recording, the speaker judged Opus 5.5 to be the world's best model based on feedback from the high-taste testers they had consulted.

“There's some stuff on the edges that aren't really about, like, you know, maybe stuff that we haven't talked about in the blog post, which is that we did do a lot of work on trying to make our API harder to distill from.”

Claim: Anthropic had undertaken work to make its API more resistant to model distillation.

“We, we've known for a long time that chain of thought extraction is one of the most powerful ways that you can distill models.”

Claim: Anthropic has long recognized chain-of-thought extraction as one of the most powerful methods for distilling models.

“And actually something like how do we preserve thinking blocks across turns and make that something that's more required is something that we've explored back for a while. And what we... where we were at was we always have this really, really hard trade-off that we have to make between user experience, like user experience changes that are maybe painful for users of the API versus protection against distillation.”

Claim: Anthropic has had to balance protecting against distillation with avoiding API changes that make the user experience painful.

“And maybe just to take a step back, the reason we don't want distillation is because the entire point of why Anthropic exists is 'cause we want AI to create positive outcomes in the world safely, and we run a lot of safeguards classifiers on the tokens that we're producing in order to make sure that they're safe.”

Claim: Anthropic's stated purpose is to make AI produce positive outcomes safely, and it applies safeguards classifiers to generated tokens to help ensure safety.

“If someone distills our models and they run those models without safeguards classifiers, there could be bad outcomes in the world, and that's something that we don't wanna have happen.”

Claim: Distilled Anthropic models run without safeguards classifiers could cause harmful outcomes.

“So we don't want people to distill the models specifically for [laughs] the sake of, um, just, like, using them for malicious reasons or for bad reasons.”

Claim: Anthropic opposes distilling its models for malicious or harmful use.

“And so it's always an interesting trade-off of, like, for all the people who are using the models for good, right, like we don't wanna make them have a harder time getting the intelligence out of the models, but we have to make some trade-offs in order to prevent that distillation.”

Claim: Anthropic must balance preventing distillation against preserving convenient access to model capabilities for legitimate users.

“And so where we landed with the last couple models is a few incremental changes to have, like, enforce these preserve thinking blocks across turns, and now, like, starting to be more about, like, across, you know, like, if you're using multiple accounts and things like this, right? Like, how do we make sure that We're, um, ensuring that you aren't able to take advantage of this chain of thought e- extraction with the models. And so we shipped a couple additional features to help with that.”

Claim: Anthropic has introduced measures requiring preservation of thinking blocks across turns and added features to reduce chain-of-thought extraction, including through multiple accounts.

“We worked actually with a lot of open source harness maintainers. We reached out to them, we talked to them, we were like, "You're gonna have to make these upgrades within your harnesses. We wanna make sure you have a good time," as well as obviously our big customers, like who are building harnesses on top of the API.”

Claim: Anthropic coordinated with open-source harness maintainers and major API customers to support upgrades required by its API changes.

“And for some of them, a really good example is they have compaction strategies that would've been broken by the need to preserve these thinking blocks. And so we shipped a new, um, compaction feature within the API to replace what they had otherwise been doing manually, but still works with preserved thinking.”

Claim: Anthropic introduced an API compaction feature that replaces some manual customer strategies and works with preserved thinking blocks.

“[laughs] Yeah, no, we, we have seen that it is harder to distill, like through the measures that we are able to, to take a look at. It has been significantly harder 'cause this is, uh, and, you know, credit to the, the researchers here, they, they did disclose it, uh, very responsibly. Yeah. Um, and so closing it down was, was great.”

Claim: Anthropic reports that distillation has become significantly more difficult according to the measures it can observe.

“And yeah, the bit I would just add to what Caitlin just said is I think, uh, at least I see a lot on Twitter or X or whatever you wanna call it, um, so it's a commercialization problem or something like that. You know, if you can steal a model and you can take this, then it's, you can run it for cheap and, and so on and so forth. While true, um, it's actually not something we really care about very much. We care much, much more about this from like a safety point of view, um, more than anything else. And so that's why we, we tend to think about it from that angle more than just like commercially.”

Claim: Anthropic's primary concern about model distillation is safety, not commercial impact.

“Realistically, if you distill from us, we, you actually pay us, so commercially we like make the money. Um, so it's not really that kind of an impact.”

Claim: Anthropic says distillation does not primarily harm it commercially because users who distill from its services still pay the company.

“Um, it's just more on like making sure that our model's like we wanna cla- we wanna ensure Claude is like good, and Claude could be derived, and if it ends up negative, that would be unfortunate.”

Claim: Anthropic wants to preserve Claude's quality because a distilled derivative of Claude could produce negative outcomes.

“So for model launches, so our team works together very closely, um, like between product and, and eng.”

Claim: Anthropic's product and engineering teams collaborate closely on model launches.

“On my side, a lot more of it is about like what is the balance of trade that we need to make between like launching something that has the right like intelligence bar, but also the, the cost to preservation. Um, like a lot of like our, our updated cash costs and things like that are indications of us being like, we want to like encourage you to take integration changes that are beneficial for actually everybody, and just like pass those savings on. Um, but a lot of these kinds of decisions around that and then how do we balance, uh, some pieces around our safety motion as well as making sure that it's like an awesome model that people can, can take advantage of. And so those kinds of traits around like I would say maybe the qualities and surrounding components around how we launch the model are more on product.”

Claim: Anthropic's product team weighs intelligence, preservation costs, safety, and user experience when making model-launch decisions.

“So we work, obviously if you're like talking about the experience of an API that's like very, very, very tightly integrated, and so it's actually been really awesome. We have a lot of engineers who actually are deeply product-minded, and we actually like love that. And similarly on my side, um, there's a lot of like product managers who are much more technical, and we want this like blended kinda interdisciplinary type of people because then they can kind of analyze it from all the different angles to try to figure out how, how, uh, to do that best.”

Claim: Anthropic values interdisciplinary teams combining product-minded engineers and technically capable product managers to evaluate decisions from multiple perspectives.

“But I tend to lean a little bit more on like commercial impact versus I think I'd actually frame it as more like the engineers are going to care much more about the integration experience. Um, so they almost take more of a UX angle to it, um, where the UX is obviously out more about the API, um, and the developer tools. Yeah.”

Claim: The speaker says product tends to emphasize commercial impact, whereas engineering tends to emphasize integration experience and developer-tool user experience.

“I think that like software engineering getting solved is, I would, I don't know if I would ever use language quite that strong. I think there's, the way that I tend to think about these things, like ambitions will continue to elevate, right?”

Claim: The speaker rejects the strong claim that software engineering is solved and expects ambitions to keep increasing.

“And maybe a good example, like a lot of what our team is spending time on right now is, of course, obviously all the things that we have to do to launch our model safely is actually the most important thing that we're doing because of all the work that we have to do on security with agents, with offline safeguards and online safeguards, all these things that are happening.”

Claim: Anthropic's team prioritizes safe model launches, including agent security and both offline and online safeguards.

“[laughs] Um, and part of that is because when I joined Anthropic a little over a year ago, we had the messages API, it was a Python monolith, we were running it in a, you know, single GCP region, and everything was fine, and none of my friends and family, like, knew what Claude was when I was going to take this job- [laughs]...”

Claim: A little over a year before the conversation, Anthropic's Messages API was a Python monolith operating in a single GCP region.

“right? And now we're in an extremely different place. And so the metrics are really wild in terms of how much volume we've seen and have had to support on the API. CPU is really constrained, uh, which has been a really interesting one for us.”

Claim: Anthropic has experienced dramatic growth in API volume while facing severe CPU constraints.

“Um, and so we've done a lot of work on how do we scale these systems, and that takes a lot of obviously just, like, distributed systems, expertise, and certain language and frameworks expertise that the models have, but when it comes to operating and evolving a system at scale like this, we're not quite- we're not nearly in a place where it's like, "Good luck, Claude. Have fun, uh, and just go and kind of make all the changes you need to make to the system, roll them out safely, and operate this thing at scale." Um, there's still just a lot of work for humans to have to do that I imagine will continue for a while. I'm curious how that's changed.”

Claim: Anthropic's systems still require substantial human expertise to operate and evolve at scale; Claude cannot yet independently make and safely deploy all required changes.

“Everything that I'm saying right now are not things [laughs] that we used to have to think about when we were launching these models- [laughs] Indeed... um, and now it's, it's the majority of the work is, like, what are we doing to prevent distillation, the online safeguards, the offline safeguards, the policies.”

Claim: Anthropic says model launches now require substantial work on distillation prevention, online and offline safeguards, and policies—work it did not previously have to consider.

“A really great example is we built a system to make sure that authorization for being allowed to use a model or being allowed to use one of these, like, programs that give you maybe, like, relaxed safeguards, that authorization threads all the way down through our system in a way that we can really trust. You know, it's not like, "Oh, you got access to Mythos," and so we throw you an API key and we flip a feature flag and call it a day. We have a really robust system on how we're doing auth and making sure that if you're supposed to be using that model, like, only you are using that model in the way that you're meant to be using it.”

Claim: Anthropic built end-to-end authorization controls to ensure model and relaxed-safeguard program access is granted only as intended.

“Like, as our capabilities get w- uh, more and more advanced and they start to spike, you know, in specific areas, that's where we need to pay, like, extra attention.”

Claim: Anthropic increases safety scrutiny as model capabilities advance, with extra attention to capabilities that become unusually strong in specific domains.

“And so a lot of that evaluation comes just from, like, those very particular cap- capabilities that we might either see as, like, offense-dominant or defense-dominant and therefore might take different strategies for all of that.”

Claim: Anthropic may apply different safety strategies depending on whether a capability is assessed as offense-dominant or defense-dominant.

“There are some more obvious ones, like between coding, right? Like, great. Go make awesome products. Go ship more things. Um, it can be really awesome for you. And then other areas like bio and cyber, it's like it could be dual use, and there could be negative effects, but obviously, of course, you could be a really valid cybersecurity researcher. You are trying to find vulnerabilities. You are trying to be a good actor. Similarly with bio, you might be researching and stuff like that.”

Claim: Anthropic recognizes that cyber and biology capabilities can enable both harmful uses and legitimate defensive or research work.

“And so a lot of these programs, like the one that you, you have up here, are to basically provide access to that dual use. You do have to specify your use case and things like that. Um, but we do want to enable the community, uh, to be able to take advantage of these capabilities and see what amazing things that they can do with it as well.”

Claim: Anthropic uses verification programs to grant scoped access to dual-use capabilities based on a stated use case.

“Um, but we do want to enable the community, uh, to be able to take advantage of these capabilities and see what amazing things that they can do with it as well.”

Claim: Anthropic aims to let the community access advanced model capabilities and explore their potential uses.

“Like, we'll see about this one in particular, but examples like this have been consistently the way that we've had to actually transform the way we develop our roadmap because, you know, in traditional product management, I guess you can be like, okay, like these are the set of things I hypothesize to happen. They will have like a s- certain sequence, a certain ordering, but you are picking bets in a way that are much more concrete versus, uh, like, you know, the de-deployment of Fable, for example. I don't think any one of us could have guessed that that would've been like a valid use case that would've happened, um, by the government and the way we just like had to go do it. But clearly the probability is non-zero. And so from this form of thinking, it's become deeply probabilistic actually, um, from like a product management standpoint of like, so should we prioritize this? When we look at things like that, we're like, okay, if we are to imagine the most crazy set of possibilities, we probably actually have very little certainty as to where things will go.”

Claim: Anthropic says product planning has become probabilistic because the company has little certainty about which extreme scenarios will occur.

“And so we actually just kind of need to like hedge and build a portfolio of bets. And so more often than not, we will take hedging scenarios of like, okay, we don't think it would happen again, but in the event that it does, we would actually want that hedge or, uh, have the ability to be able to do that very well. And so we will prioritize things like that.”

Claim: Anthropic often invests in contingency measures for events it considers unlikely but wants to be prepared to handle if they recur.

“And then we have increasingly found that more often than not by taking that strategy of product development, most of the time we actually find ourselves being like, we actually do use the thing.”

Claim: Anthropic says its hedging-oriented product-development approach often results in the company later using the capabilities it prepared.

“So you take a model, uh, call it like Opus, um, and you want to be able to use advanced cyber capabilities on, on Opus for research or for whatever you wanna go do, like you're, you're a defensive user. Um, you can apply to this program to be able to, with your use case, and if approved, what we'll do is actually diminish the safeguards on your version of the model so that you can then go ahead and try to take advantage of those like cyber activities, um, for, for your use case. Now obviously that requires like per use case understanding, it requires scoping down to a particular user, 'cause we wouldn't do that generally, you'd have to like apply.”

Claim: Anthropic's access program can grant an approved, scoped user reduced safeguards on a model for a specified defensive cybersecurity use case.

“Now, it was surprising to us, or maybe like, you know, in the portfolio approach, we found that we were actually able to take this and make it ex- we were like, when we're designing it, we're like, what kind of should it be? How generic should it be? Should it not be too generic? Should it be overly focused on making this verification, you know, style portal plus granting of, of very specific capabilities of the model? And we tended to opt for like more genericism because we were like, we don't know. Like maybe we can use this for other things, maybe not.”

Claim: Anthropic deliberately made its access-program primitives generic because it did not know what other applications they might serve.

“And then it ended up being massively useful for things like, um, you know, uh, access to the model writ large for managing, uh, any of our trusted access programs for handling de-deployments of models because we were actually very, very generic about how we handled every single primitive.”

Claim: Anthropic's generic authorization primitives proved useful for managing general model access, trusted-access programs, and model deployments.

“But here, increasingly, because we choose the more generic approach, we find ourselves reaching for that solution more often than not, and it pays like increasing dividends for that reason.”

Claim: Anthropic says its generic authorization solution is used frequently and provides growing returns over time.

“Like a trust grant means like you have been given access to some specific model or you've been given access to some specific program or, you know, like there's a, a bunch of different permutations of this.”

Claim: Anthropic defines a trust grant as an authorization granting access to a particular model, program, or other specified resource.

“Uh- Minus the safety stuff, which I think is, is the more complicated part. But outside of those pieces, there is a lot of getting feedback from our customers. So, you know, we make the model, there's a lot of safety testing. Um, there's also evaluation of where we think it needs to be. We make the required trade-offs and decisions as necessary, and then we have to go through testing. Uh, we'll cut various versions of the model, we'll evaluate it. We'll do a lot of internal, um, ant fooding, uh, is- which is what I call it, and that actually always really helps. Like, we catch like really early things like, uh, this is, you know, maybe not the best, and we obviously use Claude a lot for our work internally, so we're able to actually see early signals around stuff. Um, and it helps us detect like, you know, which, which variants might be better than others. And then when we feel really confident with like a certain set of, of variants, uh, we will cut a version for our, for our customers. Um, sometimes that will go through our first-party products, uh, like through Cloud Code or Cowork or things like that. Um, in other cases it will go through our API customers, um, and we'll engage in a kind of like a, a early access program effectively with those customers. They'll need to make changes into their harness, and we've engaged with, uh, a lot of customers like, like Cognition, like, like Hers, like these folks, and we have a bit of a process on how we handle that, um, so that we all kind of get... They always get the feedback that we, we need. Uh, does it perform well in your evals? Do you like it? What don't you like? Some things are, are vibey, and we like to hear those things too. Um, and some things are just like straight up evals, like, did it do well on the things that you care about? Yeah. And we take that all together and once we've all feel good about like all those different pieces, we launch the model. The state of model feedback is- feels very unstructured still. Mm-hmm.”

Claim: Anthropic's model-launch process combines safety testing and evaluation with internal testing and customer feedback, including early access, before release.

“And we take that all together and once we've all feel good about like all those different pieces, we launch the model. The state of model feedback is- feels very unstructured still. Mm-hmm. Basically, it's always a Slack channel between companies of like, "Hey, this thing seems weird." And then people are like, "Okay, I'll fix it, I guess. Give me an ID or something." Yeah.”

Claim: Anthropic describes model feedback as still largely unstructured, often exchanged through informal Slack channels between companies.

“Like, but I think to your point, the only customers really that are part of our early access programs are customers that have really robust evals.”

Claim: Anthropic generally selects early-access customers that have strong evaluation suites.

“Like, I do think a lot of this is vibes-based, and I think a lot of the conversations tend to be kind of like vibes-based and, and on the fly, and part of that is because we're, like, racing, right? It's like we wanna minimize as much as possible the window between we have a version of the model that we've cut for testing and we give everyone access to that model, and so it is just like this really tight window, but the customers we tend to do it with are the ones who, like, we know they're gonna go like kick off their whole eval suite and be able to come back to us with like really actionable feedback as quickly as possible alongside the vibes, um- Yeah, yeah... the vibes-based feedback. So, um, of course there's like definitely like processy things that we could do to make these things better over time, but I mean, you'd be surprised.”

Claim: Anthropic minimizes the time between preparing a model for testing and customer access, choosing customers able to run evaluations and provide rapid, actionable feedback.

“I mean, you probably wouldn't be surprised, but I think some people would be surprised about how few, you know, like products that exist in the world that are agentic that, like, don't have good evals [laughs] to be able- Oh, we had-... to help test these models. We had Boris on the pod two years ago saying we don't have evals for Cloud Code. Yeah. [laughs] I mean, that's changed, but- That is true. [laughs] That's- that's changed. But you know, like, uh, uh, yeah, a lot of things like, you know, your evals are your users, and you don't have time to formalize them, or you don't have the experience or whatever- Yeah... and that's fine. At some point when you get serious, you- you'll have evals. Yeah, totally. Exactly.”

Claim: The speaker says many agentic products lack robust evaluations, with some teams relying on user feedback until they can formalize evaluation practices.

“The APIs, uh, customers in particular- Yeah... are very useful because they'll put Cloud into a different harness. Um, and we want Cloud to be very, very general. Uh, so its ability to work in other harnesses is, is quite important to us, and so that style of feedback tends to be exceptionally useful for us because it's, it's novel, um, from what we would've gotten from ant fooding.”

Claim: Anthropic finds API-customer feedback valuable because testing Claude in external harnesses exposes novel behavior beyond internal testing.

“Uh, I think we're very frequently not surprised, but we like to look to be validated on some of those things that we might hypothesis. Honestly, the most useful feedback is negative. Um, like, you know, it didn't do well at this, it didn't do well at that. We love that feedback, and then we're like, "Great, how do we fix it?" Yeah, and it tends to be some of these things that are like...”

Claim: Anthropic considers reports of model failures especially valuable because they guide improvements.

“It's really hard because not everyone actually wants, like, Claude to land at the exact same spot on every spectrum. Mm-hmm. And, and maybe a good example is like, you know, laziness versus proactivity and taking action. And so there's sometimes when, like, we have a model and, and people are like, "Ugh, like, just finish my task." Like, "Stop being lazy," like, "Do better." And other times when it's like, "Hey, we were like just kind of feeling this out and planning, and then you like went and wrote a bunch of code, or you went and like took- Yeah... a whole bunch of action that I wasn't ready for," and getting that exactly right is literally impossible because everybody actually wants something different in reality.”

Claim: Users have conflicting preferences about Claude's level of proactivity, making a single ideal behavior difficult to achieve.

“But I do think we've had situations where we've cut an early version of the model, we found that where we saw too much laziness, right? And like, and we like put it back in the oven for a bit- [laughs]... right? And, and tried to train that back out. And so those are the sorts of things that we may not have caught if all we're doing is internally ant fooding within our products and within our API. Mm-hmm.”

Claim: Anthropic has identified excessive laziness in early model versions and retrained them to reduce it.

“Like, like it's, it like clearly- A model can do so many different things, and it's so depe- everyone has their own Claude MD. Uh, how do you eval that? [laughs] Yeah. We don't do a great job of it. [laughs] There's like the... There's... No, seriously, it's, it's really hard. It's super hard, and they're, they're competing and contrasting. Yeah, look, look at chat. It's, it's horrible. Yeah. Yeah. The constitution's helpful, but, but for things like this on how it personalizes to instructions, uh, which are different, uh, and preferences which are very, very different across these things, um, is very hard for us to do.”

Claim: Anthropic says it currently evaluates personalization poorly because users' instructions and preferences vary widely.

“We generally try to solve this by having more parameters that then allow us to tweak it per customer base, and if we don't control your customer base and you're an API customer, then we try to give you recommendations, and then you try to do that tweaking.”

Claim: Anthropic addresses personalization partly through configurable parameters and recommendations that API customers can use to tune behavior.

“It doesn't exact- It's not exactly at the same level, like personalization to the Claude that, you know, various Claude MDs or now Agents MDs or whatever, but we do try to give things around the layer of like level of effort to like how much effort you should put in, like what are, uh, the, like the level, like maybe the size of the model that you might actually like try to care about, things like that. But it's... I think it's, um, it's not quite there for what we would actually need for like that level of personalization.”

Claim: Anthropic says its existing parameter controls fall short of the personalization it ultimately needs.

“Um, so it's a very hard problem, and right now, a lot of our way to solve that is beyond the parameter, like the giving multiple parameters to something, is try to exhibit conflicting taste, honestly, a- across like all of the feedback that we get and try to ch- try to make like a judgment call on some of these things. I think over time they actually... and as you scale up more customers and more users, that's like not a very scalable way to go do something.”

Claim: Anthropic currently resolves conflicting personalization preferences through judgment calls, a method the speaker expects to become unscalable as users increase.

“Um, and we will need to develop ways for Claude to kind of like take in your personalization, um, and maybe that's like a signal you can provide or something you could tune on, or things like that. Um, but I think we'll have to like take a route that's much more oriented, uh, towards that direction.”

Claim: Anthropic expects to develop mechanisms for Claude to incorporate user-specific personalization, potentially through user-provided signals or settings.

“It's a Claude code setting. They default it to medium. Yeah. They just- Yeah. You can turn it on, right? [laughs] And, and this, and this thing varies. This thing does... Like, this is an, this is a perfect example of a parameter that we use to kind of like personalize. I, I would say that's maybe a little too aggressive of a term, but we kind of like customize maybe for different sub-segments. So like, uh, the default setting will adjust for, uh, whether you're a free customer, a prosumer customer, Claude code versus, uh, co-work versus enterprise, like things like that, versus API and stuff like that. Um, knowing that you can obviously always adjust it, but those are kind of like light ways where we try to give that.”

Claim: Anthropic adjusts default effort settings across customer segments as a limited form of customization, while allowing users to change them.

“One thing that we are hesitant to do is we don't want people to feel like they can't control their customization of Claude.”

Claim: Anthropic wants users to retain control over their customization of Claude.

“Uh, so we could take a heavier hand on these things, for example, but we want to give users the ability to experience the intelligence levels of Claude as they kind of see fit, especially if you want to go towards like the higher end, or you want to experiment with maybe sometimes in the, in the lower end if that actually is just like faster for your use case, which is a hard product trade-off I think. Um, and so we, we tend to lean towards gentle defaults but give you options to, to adjust, which I think works okay-ish for now. But I imagine us, you know, like learning how to do that better over time for customers. We're not like super, super happy with any of this yet, but we know as we launch a model, like y- you asked like what does it take to launch a model?”

Claim: Anthropic currently favors mild defaults and user-adjustable options, but expects to improve its customization approach over time.

“you wouldn't really get very far, and that's one of the things actually that, um, is part of the value prop with something like Claude Managed Agent, which is like still programmatic access to Claude in an API, but you get a harness out of the box where we do every time we're launching the model a lot of the work to go figure out like what are the new parameters, what are the breaking changes, how do we get the best out of Claude within our harness, so that for the people who are using it, um, you just kind of get that out of the box.”

Claim: Claude Managed Agent offers programmatic Claude access with a ready-made harness that handles model-specific parameters and breaking changes for users.

“But give, give the quick three. Yeah. It, it depends on your use case, but, um, so Claude Managed Agents is our API today. We have the Messages API, which is essentially raw token access. Then we have Claude Managed Agent, which is a suite of APIs, but it lets you define an agent, start a session with that agent, um, and it's essentially just kind of a higher order abstraction that lets you skip setting up all the infrastructure, setting up your harness, and doing a lot of that type of work. It's a much more out-of-the-box integration for people who want to programmatically run agents. Does CMA, uh, share infrastructure with Cloud Tag?”

Claim: Claude Managed Agent is an API suite for defining agents and starting sessions without building the underlying infrastructure and harness.

“We also kind of run all of our foundational product infrastructure and product services. So, like, a really good example is, like, our team runs our sandboxing fleet that backs both Claude Manage Agents as well as products like Cloud Tag and otherwise. But at the actual harness layer for those two products, like, there's just differences in what we might wanna do within the harness so that they don't, they don't actually share a harness. Um, but they share a lot of the infrastructure that powers the execution and, um, powers the APIs. I think that's, that's really... Like, people should use this more.”

Claim: Claude Managed Agents and Claude Tag use shared sandboxing and platform infrastructure while maintaining distinct harnesses.

“You know, is, is it still, still ongoing? Like, what's, what's the emphasis here now? Managed Agent specifically? Uh- Memory, dreaming. Dream... Yeah. Oh. It is a big part of what we're doing. I think maybe the way to think about it is we launched Managed Agents as a platform for people who, like, didn't wanna do all these things themselves, right? Or just wanted to, like, offload, like I'm getting the best out of Claude because our team is literally sitting here, like, eval-ing all these things and making sure that the harness is gonna be really powerful. Memory and dreaming is all about just, like, how do you make sure your agent can continuously improve over time? And all of that is still an important focus for our team.”

Claim: Anthropic continues to prioritize memory and dreaming as ways to help agents improve over time.

“But what I would actually say is, to the point of what we talked about earlier, as launching models has become harder and harder from a safety perspective, I think most of our team's energy is going less into like, "Here's a fun feature to, like, hill climb on this agent or whatever," and much more into, "How do we bring safety into all these things?" Because the most important thing that we can do is get the power of the model out there and get it out there safely.”

Claim: Anthropic says its team's focus has shifted from agent-feature experimentation toward safety as model launches become more difficult to manage safely.

“I think compared to maybe earlier this year, we have slowed down on, like, skills, dreaming, memory, like these kind of, like, moment in time injections. So you dropped your skills? No, no, no. We love skills. We love skills. [laughs] Um, we, we are making, we are making updates. Um, I'm not saying that we're not doing anything, but, um, we, we love skills as well. But yeah, so I think we've slowed down a little bit comparatively, um, for some of the other efforts.”

Claim: Compared with earlier in the year, Anthropic has slowed some work on skills, dreaming, memory, and moment-in-time injections, while continuing to update and support skills.

“But, uh, we are very bullish on, uh, people kind of doing some level of hill climbing at the harness layer, uh, to, you know, some of the previous conversation here. We do a lot of this ourselves, but the harness that we provide is very simple in a way. It's, it's intentionally simple. Uh, we don't want to be actually overly opinionated.”

Claim: Anthropic supports developer-led optimization at the harness layer and intentionally keeps its own harness simple and minimally opinionated.

“Now, the interesting part on top of that, though, is, like, once you get, get that base layer, there are these, like, hill-climbing strategies that we construct, and, like, dreaming is actually one of them. So dreaming under the hood is a multi-agent system that we hill climbed on top of CMA, and it's actually ag- CMA on top of itself, basically. CMA on top of itself. It's, it, like, spins up another- Yeah. Yeah. Uh- Okay. Uh-... the way dreaming works is that we spin up another managed agent that then can expect the previous managed agent and then write, uh, memories to that managed agent. So it's managed agents all the way down.”

Claim: Anthropic's dreaming system uses nested managed agents: one agent evaluates a prior agent and writes memories back to it.

“Uh, in the same way that we did, we built dreaming that's, like, managed agents all the way down. There's a update that we'll make relatively soon, um, where we're updating the underlying primitive. We'll expose it, uh, to allow users to kind of, like, construct harness strategies maybe, or however you wanna call this stuff. Like multi-agent strategies. Multi-agent strategies- Yeah... that allow you to kind of build your own version of this, and we'll give you, like, you know, templates of, like, this is how we started with dreaming.”

Claim: Anthropic plans to expose a primitive for building custom multi-agent strategies and provide example templates based on systems such as dreaming.

“It's less that we're slowing down on this sort of stuff and more saying, like, we've got the primitives in place. We can continue to let people just, like, take and run with those things while giving them the tools to say, "Here's some, like, cool multi-agent strategies and, like, things that we've put into the, into Claude Manage Agents to make it really powerful." Now you can, like, let Claude do work to take advantage of those primitives and make it even more powerful yourself, like create new multi-agent strategies and whatever it might be, while a lot of our team's attention turns to, okay, great, like, community, run with that, right? And, and you do you while we continue to focus on what are all the things that we need to do to make sure that you get the access to the models within that system safely.”

Claim: Anthropic plans to enable community-built agent strategies on its primitives while concentrating its own effort on safe model access.

“There's a lot of different types of evals, right? Like, not all evals are made equal for different, you know, like, situations that you're in and things that you're trying to test for. You obviously have just, like, static deterministic stuff, and then you have LLM as a judge. You have all these different strategies. But I guess I would just say, like, what we found from most people is, like, you probably need more rather than less.”

Claim: Evaluation needs vary by situation, but the speaker's general advice is to build a larger rather than smaller evaluation suite.

“[laughs] Um, it's just, uh, there's not, like, one perfect, like, couple sets of evals that you could put together that are gonna get you all the answers that you need to make your system better. We found that people end up needing, like, a relatively large suite of these things, and then it's like you can have your agents work with you to evolve your eval suite over time as well.”

Claim: No small, universal set of evaluations can answer every question needed to improve a system, so teams generally need a broad evaluation suite.

“We found that people end up needing, like, a relatively large suite of these things, and then it's like you can have your agents work with you to evolve your eval suite over time as well.”

Claim: Agents can help teams iteratively develop and improve their evaluation suites.

“Um, but I think it's just, like, n- underinvesting is maybe, like, the biggest mistake that we've seen from people on that front. Yeah. And we start simple.”

Claim: The speaker considers insufficient investment in evaluations one of the most common major mistakes.

“I think some people, like, go a little crazy, and they're like, you know, they, like, look at all the, the menu of options- Yeah... and you kinda list, and you're like, "Wow, this is so crazy. Let me try this and that, and what's the best one?" And, like, while they do all those things, like, the number of evals they actually have are, like, zero. Yeah. So you might as well, like, do something simple. It will help. Um- Yeah, I think some, some part of it, people are inexperienced, they're not trained to do this, um, and they don't know what good looks like. Totally.”

Claim: Teams should begin with simple evaluations instead of delaying evaluation work while searching for an ideal method.

“type models is we're treating engineering like research, and when you treat your engineering like research, you naturally do more evals, you do more labeling, you do more scaling of runs, or, like, prototyping and scaling of runs. And you're not training a model, you're building the harness, the product, whatever it is, but it... you're approaching it like research.”

Claim: Approaching engineering as research leads teams to define goals, run evaluations and labeling, and scale experiments even when developing products or harnesses.

“You're defining upfront what your, uh, goal is, trying to formalize it as an eval, and then, uh, try to identify, okay, why, why is it failing? And sometimes your eval is wrong. Totally. Actually, a lot of the times your eval is wrong. Yes. Sometimes your environment's wrong, and then sometimes the model's wrong. Yeah.”

Claim: Research-oriented engineering starts with a defined goal and evaluation, then diagnoses failures by checking the evaluation, environment, and model.

“Um, and we'll continue to launch more features within Claude Managed Agents to give people more tooling to do this sort of stuff out of the box. Yeah. And you'll probably find from us that you'll get something that's like, "Here's a product that will help you create evals for your managed agents," or whatever it might be, and it'll feel very much like working with Claude [laughs] in order to do that.”

Claim: Anthropic intends to add Claude Managed Agents features that make evaluation workflows easier to perform out of the box.

“And then latency, I think we saw somewhere between, like, 10 and 20%, um, improvement on, like, time to first token specifically. So a lot of just really cool improvements, uh, across the board. It's higher on, like, uh, longer running, like, workloads, but- So it'll, it'll roll out- Yeah... um, soon. Yeah. But it'll get faster. Do you fight the, the urge... You know, I've been involved in some major rewrites.”

Claim: Anthropic reports that its Rust Messages API improved time to first token by approximately 10–20%, with greater improvement on longer workloads.

“Is it, is it a pure rewrite? So we actually, we did. It was a pure... It was, um, maybe the thing that I would call it is, like, a port. What we did was we, we did exactly that. We, like, resisted the urge- [laughs]... on all these things. Then, a- and because what the team already had to keep up with was little, like, SWAT team that we put together to go and do this was doing this while the whole rest of our team was, like, building features into the API, right? And they did all the, like, static analysis stuff. They would yell at you because you built a feature over here and you didn't consider Rust over there, right? And, like, they dealt with all of that. But they resisted the urge to add features themselves or, like, make the system different.”

Claim: Anthropic kept its Rust migration focused on porting the existing system, resisting feature additions and behavioral changes.

“Now, their next goal is to go and do some sort of re-architecture type work, including stuff that I think will be more user-facing.”

Claim: After completing the Rust port, Anthropic plans further architectural work, including potentially user-facing changes.

“Like, we're starting to roll out some things that are like, you know, what if we were able to not have you have to, like, pass back and forth the entire, like, conversation state, right? And, um, if you didn't have to do that on the messages API because you were open to, like, you know, a storage of that state for some period of time, right, then, like, especially for long conversations where the context really builds up, like, you get a lot of really great latency and efficiency wins there, too.”

Claim: Storing conversation state in the Messages API could eliminate repeated transmission of full conversation histories and improve latency and efficiency, especially for long conversations.

“right shape of, like, how we should run our compute- Yeah... in order to power all these sorts of things. Yeah. Um, but it's also interesting because we can't necessarily rely on, like, because we get our compute across. We run on all three clouds. We're running all over the place. Like, we don't necessarily have, like, one shape of here's exactly, like, the compute, the types of machines that we know we're gonna run on and the way that some certain compute works that we can, like, rely on across the board, if that makes sense, other than, like, the most vanilla versions of these things.”

Claim: Anthropic operates across three cloud providers, limiting its ability to depend on a single standardized compute configuration across its infrastructure.

“We put out this blog post at the time of launching where we talked about we have the brain separate from the hands, and what that means is, like, the harness runs on a durable server, really ours, right? And then- Oh, wow, neat. Nice, nice pull. Nice pull. Yeah. No, yeah. That's great. Um, and the hands are, it's like a- Perfect... execution environment, like a sandbox or some sort of container, right?”

Claim: Claude Managed Agents separates the agent's durable server-side harness from its sandboxed or containerized execution environment.

“The part of the reason why we did this was for, like, latency and reliability and these sorts of things, right? Like, if the harness is just running on a durable server and it's only spawning a sandbox or a container to execute when it needs it, that means we can get you tokens back really quickly on your initial request. It means that your sandbox can die, but, like, your whole agent doesn't die because the harness is durable and over here and the har- you know, the agent's storing state separately also.”

Claim: Claude Managed Agents' durable harness can return initial tokens quickly and preserve the agent when its execution sandbox terminates.

“And there was a whole product development journey that we went on to get there because our first iteration of an API for agents was just, like, throw cloud code in a sandbox and, like, shove an API on top. And the reason why we didn't launch that as our, like, durable scalable solution is everything that I just said. It was like you've got to boot a sandbox and boot cloud code before you can even get a token back, right? The sandbox dies, and you lose your whole agent, and so reliability wasn't where we wanted it to be. So we architected this way mostly for reliability and latency. But the interesting thing, there's actually a, a...”

Claim: Anthropic rejected an early sandbox-based agent API design because it delayed initial responses and lost the agent when the sandbox failed.

“But one of the more important properties of this architecture is you have credentials for any, any work that the agent needs to execute, right? Like, if you're gonna go talk to an MCP server or if you're gonna go, like, use a CLI or call some API, whatever you might wanna do, right? And, like, you need to authenticate those things. You need to store those credentials somewhere so that the agent can use them. And the way that we architected this was we have a totally separate vault where those credentials are getting stored, and as once the agent decides, like, I'm gonna go and call out to one of these services, the agent only ever sees, like, a placeholder, right? And, like, on egress, we have a proxy that goes and, like, pulls the right credential out and injects it and lets that out.”

Claim: Claude Managed Agents stores credentials in a separate vault, exposes only placeholders to the agent, and injects credentials through an egress proxy when requests leave the system.

“And so what's interesting about that is, like, the agent is never, like, in the same place where the credentials are, right? Because in the old world of, like, just boot the agent in a box and, like, let it run, you had to send credentials in there, and then you're, like, doing some sort of, like, ah, how do we, like, scrub this and that and make sure that, like, the agent can't take that credential and go do something that it shouldn't be doing. It is just, like, literally a, like, literal infrastructure wall [laughs] that says the agents, like, can't access those credentials and can't do anything bad with them. So that was one thing that was really great.”

Claim: The architecture creates an infrastructure boundary that prevents agents from directly accessing stored credentials.

“The other nice thing is, like, on that sandboxing infrastructure, we can make sure that we have really great egress controls. So we have, like, domain allow lists and things like that, or you just, like, totally shut off network access so that you can't say, "Okay, agent wrote untrusted code. That code executes in sandbox, and that code can, like, reach out to the internet," right? Like, now it only can reach out to where you say it can reach out to.”

Claim: Anthropic's sandbox controls can restrict network egress to approved domains or disable network access entirely.

“So there's a lot of really great properties around all of this within the architecture and within the way that you can configure your sandboxes that make it so that you end up, like, secure by default.”

Claim: Anthropic says its agent architecture and sandbox settings provide security by default.

“I think the two most important things are architecture, so that it's, like, literally impossible for an agent to do something that isn't okay, and the configuration that you have on those sandboxes.”

Claim: Preventing agent sandbox incidents depends primarily on architecture that makes prohibited actions impossible and on correct sandbox configuration.

“And so one of the really, really nice things about Cloud managed agents is you just, like, get that out of the box, and you get something out of the box that, like, as we learn more and we continue to evolve the properties of those systems, you kind of get that for free.”

Claim: Claude Managed Agents supplies security controls by default and passes along improvements as Anthropic evolves its infrastructure.

“The second thing I would say is just, like, everybody's figuring all of this out, like, relatively newly because, you know, long-running agents is a relatively nascent thing that everybody is building and trying to deal with. And so everyone's figuring out some of these things on the fly and just racing to put the right, you know, like, controls in place and, you know, some of that is just happening as we go.”

Claim: Long-running agents are a new field, and organizations are developing operational practices and controls as they build them.

“So I would say it's not, you know, we're- people are bad at this, right? I think it's just more it's a nascent field, f- people are figuring it out, and I think when we talk about things like pacing the frontier, the way that we think about it is really specifically we're doing all the work that we're talking about right here for all of our agents internally and making sure that all these things are accessible to all of our customers so that, like, by default, the right guardrails are just in place so that you don't end up having agents doing things that you don't want them to be doing.”

Claim: The speaker attributes agent-sandbox incidents primarily to the field's novelty and ongoing learning, rather than simply to incompetence.

“I think it's just more it's a nascent field, f- people are figuring it out, and I think when we talk about things like pacing the frontier, the way that we think about it is really specifically we're doing all the work that we're talking about right here for all of our agents internally and making sure that all these things are accessible to all of our customers so that, like, by default, the right guardrails are just in place so that you don't end up having agents doing things that you don't want them to be doing.”

Claim: Anthropic's approach to pacing the frontier includes building internal agent guardrails and making them available to customers by default.

“um, it's like we are continuing to evolve more and more and more safety properties in place, again, across these, like... I kind of am starting to think about this in these three dimensions. It's like the actual, like, architecture and infrastructure security, then it's the online safeguards that are running when a model is doing its work, and then it's the offline safeguards classification that's happening on data at rest.”

Claim: Anthropic's safety framework has three dimensions: architecture and infrastructure security, online safeguards during use, and offline safeguards that classify stored data.

“So across all those three dimensions, we're putting all the right work in place to make sure that we feel really good that we can put a model out there and it's going to be safe, and really, that's fundamentally what we mean. And so for Opus 5.5, it's an excellent model. It's also really, really great on all the dimensions that we care about across those three things, and therefore we feel good about releasing that model.”

Claim: Anthropic says Opus 5.5 satisfied its architecture, online-safeguard, and offline-safeguard criteria, supporting the decision to release it.

“So we went for what we thought, what we were hoping is the best possible user experience within this while we continue to hill climb on making those false positives better, but it's obviously, like, it's still a frustrating experience for people when you're trying to work with a model and then you fall back to a different model, and so it's just something we have to keep working on.”

Claim: Anthropic continues trying to reduce false positives because safety-triggered model fallbacks create a frustrating user experience.

“So going back to that bio homework scenario, while we may have classifiers that have detected that it is possibly bio- I'll also say, like, the, on the API, we went with a spectrum of solutions for this for customers. So first of all, like, if you've built something on the API and you run into a block, like, you don't have to, like, continue the conversation on a fallback. That's what we've done in our apps, and we've made that a feature of the API, and you can do it, like, server side within our API. You can choose to, like, set, "I want to do the fallbacks," right? And, like, we'll do that. We also have it within our SDKs on the client side. There's helpers for if you want to, like, manage fallbacks in various ways. Or you could choose to, like, re-prompt differently in some way, right, and, and continue with the same model.”

Claim: Anthropic's API and SDKs let customers control fallback behavior, including disabling fallbacks, managing them themselves, or re-prompting the same model.

“[laughs] Um, but I do think that there's some in-between interesting solutions that we'll continue to experiment with to figure out how we unblock people and, and do so safely, but with a good user experience. Yeah.”

Claim: Anthropic plans to test additional approaches for safely unblocking users while preserving a good user experience.

“Like, with the router you, you go through, it's like, let's say you want to do a cost router, right, for the task complexity that you have. Makes sense. Uh, very hard to do. Uh, you don't know everything upfront about your task, so you can't, like... Even with perfect, you just don't actually have perfect information. Even if you could actually understand each model perfectly, which you don't, but even if you did, you also don't have perfect information about your task. And so there's, like, a lot of error associated with your, your routing logic that you would have to create, whether no matter how good a classifier you produce. And yeah, maybe it's, like, a super big brain model type of problem, but regardless, it's I think a very difficult one.”

Claim: Model routing is inherently difficult because task requirements and model capabilities are imperfectly known, producing errors even with strong classifiers.

“You could expose all that logic to customers like, "Hey, you know, your task was to center a div, so I'm gonna route you to something super cheap." Uh, and like, then this other one's, like, detect vulnerabilities, so I'm gonna route you to something, like, really expensive. Um, if you expose all of that, I don't think that's, like, a great user experience ultimately either.”

Claim: The speaker doubts that asking customers to manage detailed task-to-model routing would produce a good user experience.

“And for that similar class of problem, I think the more idealized UX, uh, would be something along the lines of, like, dynamic, like, effort or dynamic, like, understanding of the level of complexity of your task, which you might actually evolve over time as you learn more and more information about the task.”

Claim: The speaker favors a routing experience that dynamically adjusts effort or task-complexity estimates as the system learns more about a task.

“But in that world, if you're kind of, like, moving between a lot of different models, it actually becomes very, very disjointed. And so, like, maybe a more unified approach would be something like it's a consistent model, but, um, it's, like, very, very low effort level or something like that. It's actually very, very cheap and very, very affordable, but it can scale itself up very, very continuously, but you have, like, a very, very consistent persona towards that model. And obviously speaking about this in the context of, like, you know, like our family of, of models. If you try to even do that across other family of models, other model providers, like the personality, the disjointedness, like, starts becoming very, very complicated. I don't know how you solve it in that layer, but at least within, like, our family of stuff, like, I'm more bullish on things that are just more consistent, and those might be ways to solve it, uh, more robustly for customers.”

Claim: Within Anthropic's model family, the speaker favors a consistent model that can vary effort and cost over a disjointed experience that switches among models.

“And like we discussed kind of at the earlier part of this conversation, we are seeing workloads, uh, change quite a bit. I think, like, in some workloads, intelligence does seem to be increasingly saturated, which is, like, a, a new, uh, situation. Um, and it's, it's getting to the place where I think there are some class of tasks that are still very, very unsaturated.”

Claim: The speaker observes that model intelligence is approaching saturation for some workloads, while other categories of tasks remain substantially unsaturated.

“I actually think there are more than maybe people anticipate, but they're increasingly going into the direction of you get that alpha, um, or that return by doing extremely long-running agentic tasks on things that historically you were not really able to brute force, but maybe you can with a lot more intelligence behind it.”

Claim: The speaker believes many unsaturated opportunities remain in long-running agentic tasks that apply greater intelligence to problems previously impractical to brute-force.

“Um, and those workloads are really, really fascinating to us, and we see a lot of value in that kind of area, both from businesses getting that value from themselves as well as, you know, like new companies or startups or however you want to frame it, um, who are interested in building kind of like agentic products on top of it.”

Claim: Anthropic sees substantial potential value in long-running agentic workloads for both established businesses and startups building agentic products.

“There's obviously a lot of, like, enterprise, uh, innovation, acceleration kind of stuff, um, but they tend to take the shape of automating a very distinct and specific process. And I think in some areas, that's actually really starting to work, which is awesome, and some areas it still isn't, so there's still a lot of, like, room to, to kind of evolve and push the frontier for those models.”

Claim: Enterprise AI applications often target specific processes; this is working in some domains but remains ineffective in others.

“And then there's, like, this new class of workload, like the, the bio stuff that we talked about or some of the cyber stuff. Like, that's a... It's just a very, very complicated set of, like, I would say, hard engineering or hard science problem. Um, and those workloads are increasingly showing up as well for us as, like, a really interesting space to just deliver value to customers.”

Claim: Anthropic sees biology and cybersecurity workloads as a growing opportunity involving complex engineering and scientific problems.

“If, like, you should use Claude to help you, of course, right? But if you use Claude to, like, totally generate a thing, you didn't spend any time on, like, actually reading it yourself and making sure that it was something of high quality, then you're putting a whole extra burden on the consumer [laughs] of that document or of that code or whatever to spend a whole bunch of time trying to parse it, and that's just, like, insane and totally unfair. But what's funny is we, you know, we push back, we push on our team like, "You can't do that," right? Like, if you're producing something, use Claude, sure, whatever, but, like, you have to make sure that it's a high-quality thing.”

Claim: People who use Claude to produce work should review its quality, because passing along unchecked output imposes an unfair burden on its readers or users.

“Like, if you're producing something, use Claude, sure, whatever, but, like, you have to make sure that it's a high-quality thing. But then we get into these, like- Yeah, you as the human are still responsible for the quality. Right.”

Claim: Using Claude does not remove the human user's responsibility for the quality of the resulting work.

“You're still responsible for the quality, but then you get into this really funny thing where because of the Claude-isms, even just like silly, like, I don't know, there's like an em dash in the middle of the thing, right? Like, people's brains trigger on this like, "Oh, no, I'm reading something that was written by Claude and not a human, and I'm, like, annoyed because I wanna be r- like reading a thing that was, like, fully produced by..." Not necessarily fully produced by a human, but, like, that has the quality bar that the human would hold, right? And so then we get into funny situations where maybe actually, like, the person did spend a lot of time making sure the thing was high quality, but they used Claude to help them, but tho- there's those little hints of Claude, and then the reader is, like, turned off, you know? Mm. And, like, that's, like, unproductive as well.”

Claim: Readers may reject work that contains recognizable Claude-isms even when a human has invested substantial effort in ensuring its quality.

“And so what we have kind of tried to do is, like, set a bar within our team that's very much like use Claude to help you because it would be really silly if you didn't use this hilariously powerful agent to speed you up in your job and in your life or to just, like, help you produce better things. But it has to, like, have a certain bar, and then for the consumer and the reader, if, like, it really actually is just slop, like, send it back. [laughs] Right? Like, just say no, just send it back. Like, you're not gonna waste your time on this. But also don't be, like, too nitpicky about the little Claude-isms, right? Because, like, we have to find the right balance here.”

Claim: Anthropic encourages Claude use while requiring high-quality output and rejecting work that is genuinely low quality.

“Like, but, but I do think that where, where we've gotten to is a pretty good spot where, like, you know, humans play a large role in making sure, like, designs of systems are good, right? And, like, implementation tends to be Claude, right? And, like, we're in a pretty good spot where for, like, a lot of the code that gets written, like, all you really need is just, like, a quick stamp, and you can keep going, and, like, you don't necessarily have someone, like, deeply reviewing every single line of the code 'cause you care more about, like, the overall plan, the overall, like, design is in good shape, less so than, like, the exact perfect implementation, and obviously rely on a whole bunch of verification to make sure the thing that you're gonna ship is high quality. And so it's actually been, like, less of a conversation these days on, like, code quality being slop, in part because Claude's just, like, gotten better, but in part because we have a lot of the right verification in place to make sure that, like, you know- Okay...”

Claim: Anthropic's engineering process emphasizes human system design, Claude-assisted implementation, and verification of the final product rather than deep review of every code line.

“I think, do you remember what he said to me? Uh, Tariq, Tariq tweeted this. The right way to use model capabilities is not to ship 10X more features into prod. Uh, spend more time understanding your users, trying experiments, building whatnot. But basically, you know, you could very easily just keep shipping random slop, but you, you should actually think about it. You should spend more time kicking off that first prompt. I know now today he's like, "Kick off plan mode. We don't need it anymore." [laughs] But, uh, outside of plan mode, you should think about what you're doing, right? It's very easy to put out a lot, and you can get product slop.”

Claim: The speaker argues that AI capabilities should support user understanding and deliberate experimentation rather than a tenfold increase in feature output.

“Yeah, I mean, I, I tend to kind of go on this rant about there's a lot of just, like, fundamentals about how software gets, like, and products get planned and shipped and otherwise that, like, doesn't need to change just because agents are, like, helping speed up certain parts of the process, but there's this inherent, like, almost laziness that's starting to come out from a lot of people where it's like, "Oh, I don't have to, like, think about some part of the process at all because the agents are doing it and helping me or whatever." But, like, those are still, like, incredibly foundational parts of the process that, like, should be things that you're spending your time and energy on, and I think that's kind of the spirit of, um, what he's getting at here.”

Claim: Teams should retain core software-planning and product-development practices even when agents speed up parts of the workflow.

“It's just, like, as a founder or cr- creator, let's call it as well, um, it just feels like because the cost of experimentation or exploration is so low now that I should be doing more of that, and he's basically saying to not do that, to, to, like, use it in service of a, a very intentional goal that I start from the start.”

Claim: The speaker recommends using cheap AI experimentation to pursue a deliberate goal defined upfront, not to generate work indiscriminately.

“So, like, the blog post of today was two weeks to make Claude AI faster. There were 12 things. Uh, we hit-- there were 13 things. We hit 12 of the 13 targets in three days, right? Well, you could just stay, okay, it's a two-week sprint, we'll do the 13, but instead there's 20 things you push bound. This is not product slop. It's not new products. It's refactoring.”

Claim: In a two-week performance-improvement effort, Anthropic completed 12 of 13 targets within three days.

“Um, I actually, I don't think I have a favorite, honestly. Um- Oh, come on. No, I really don- uh, okay. I guess maybe I like the one that you say, where's half doing something is expensive way of not doing it. Yeah. Maybe that's my favorite. That's, that's very on topic for us today. Okay, nice. [laughs] [laughs] Nice. Love it. It's, uh, yeah. But I, I guess, uh, y- you know, you, you obviously you're in a different job now- Yeah...”

Claim: The speaker favors the maxim that partially doing a task can be an expensive substitute for not doing it at all.

“So this was a very long time ago. Um, I went through a period of my life where I was like, reading constantly, actually. I found myself like, very bored, um, of the stuff that I was doing, and I felt like I was just like, not being like, intellectually challenged. And so I actually went on as a bit of a bender where I, I think I read something like 200 or so books in like a year. Um, and I was just curious about like every... I just wanted to like read everything.”

Claim: The speaker says that, during a period of boredom and insufficient intellectual challenge, they read roughly 200 books in one year.

“And at the time, it really wasn't to build like a Twitter follower- [laughs]... or anything. I was just like tweeting into the, like abyss. There was like... I have like, no followers. These were like tweets for myself, and I was like practicing a little bit of like, how would I try to compress the idea that I have in as few words as possible. It was kind of like the personal test I had for myself.”

Claim: The speaker says their early tweets were a personal exercise in expressing ideas concisely, rather than an attempt to gain followers.

“I gotta ask, after all the reading, any book, book recommendations- Yes... blogs, long articles? My favorite one still is probably The Black Swan by- Nassim Taleb... uh, Nassim Taleb. Um, that was like one of those like, I was like ki- to me at least, it was like a bit of a, like a life-changing philosophical book where I was like, that's a completely different way to look at the world.”

Claim: The speaker regards Nassim Taleb's The Black Swan as a favorite book that substantially changed their worldview.

“And the concept- You know, I was thinking of Black Swan when the Fable- Oh, really? Yeah... Fable thing happened, right? It's a thing that you didn't know could happen. Very much so. Now that it happens, like it's a, it permanently changed your framework of- Exactly... how things work.”

Claim: The speaker says the unexpected Fable deployment permanently changed their mental model of how events can unfold.

“Not intentionally. I think it's more just me somewhere trying to articulate myself. But I think that's the core success- All the time, yeah... of succeeding in topic, is like you have to be very good on Slack. Yeah. Oh, you do have to be very good on Slack. That's actually true. Right? Um, it is, it's a pretty intense place, but I think I'm a little... So I'm kind of psycho about like inbox zero on both email and Slack.”

Claim: The speaker believes strong Slack communication is important for success at Anthropic.

“Um, but I like pretty aggressively prune like here are the channels that I'm like actually in and paying attention to, versus the ones that are like muted below the line sort of thing, and I'm like always making sure that I'm on top of the ones that I need to be on top of. But it is, it's basically, it feels like a full-time job. Like there are days where I'm like, I could cancel all my meetings and write no documents, do no work, right? And literally just sit there like on Slack the entire day, and I would have done what felt like an entire day's worth of work just by like engaging on Slack with all the things. Yeah. It's pretty wild.”

Claim: The speaker manages Slack by muting low-priority channels and focusing on essential ones, while noting that Slack engagement can occupy a full workday.

“I feel like you do all this e- you put in all this effort, and yes, like you, you met the mark on like responding to people and getting certain things over the line, but you didn't actually produce the output that you wanted to. And like that's like a, that's like personally a very hard thing for me to deal with. Yeah.”

Claim: The speaker finds it difficult when a day of communication and problem-solving meets immediate needs but produces none of the intended individual output.

“So I try to be like, I need to like dedicate time to like actually thinking and actually like trying to produce something in order to feel like I'm like actually being productive. And ignore s- and drop stuff, right? You, you're gonna drop stuff. Um, yes. Right? I have gotten a lot of feedback. I'm a little hard to reach on Slack. Um- Mm. Oh, yes Yeah, pretty terrible thing that happens now- I've been exchanged. Exchanged...”

Claim: The speaker deliberately sets aside time for focused thinking and output, accepting that some messages or tasks will be missed.

“Um, but I do think actually the, the types of Slacks that I'm like when I'm pruning through and getting are often just like our team is like blocked on something 'cause there's some hard decision that needs to happen or whatever it is, right? And it's like, it is just like a form of, whether it's in Slack or some other way, right? Like it's a form of getting the team unblocked, like executing in high quality, that is my job anyway, and it like happens to happen a lot in Slack. But I do agree that it ends up feeling like, what did I even produce, when you get through a whole day worth of literally just like going back and forth with people and trying to solve problems and- You, you produced a, a functioning team.”

Claim: The speaker considers resolving blockers and enabling high-quality team execution productive work, even when it occurs through Slack rather than creating a standalone deliverable.

“And when I joined, um, we were like, "Okay, we like need to grow up a little bit and need to figure out how to like engage with the community, engage with the world, and, um, we need to like really invest in this devrel function." And um, we ended up spinning up a devrel team within platform and a devrel team within Claude Code, in part because it's like two pretty different, like developer ecosystems that we're trying to engage with across the API business and agentic coding, right? And so we ended up doing it as like two separate teams, but kind of like dotted line to each other and spend a ton of time together.”

Claim: Anthropic organized developer relations into separate platform and Claude Code teams to address their distinct developer communities.

“But really specifically, like what's hard on platform is platform in some- sometimes in our like public narrative, like our platform business is a really large business, but it can kinda get lost in between research and policy and all this work that we're obviously doing that's like our most frontier important work, and then our applications like Claude Code and Cowork and all the things happening over here. When you think about Anthropic and you think about our public narrative, those are some of the things that often come to mind, and less so that we're running this like massive developer platform business that's really important and so many people are building on top of.”

Claim: Anthropic's platform developer-relations team works to raise awareness of the company's large developer-platform business, which can be overshadowed by research, policy, and consumer applications.

“And so a big goal that we have with our devrel team within platform is figuring out not only like the really classic like how do we better engage with like, you know, product, like what people are building and how we get feedback back into the product and these sorts of things, but a lot of it's just like finding our voice- Mm... and finding what our brand needs to be and what our narrative needs to be and like who we even are gonna be in the community and not kind of get lost in all of the things that are happening.”

Claim: Anthropic's platform developer-relations team seeks both to feed developer feedback into the product and to define the company's voice and identity in the community.

“And so that I think has been interesting 'cause I think the brand that we've started to spin up for our devrel team, in part because actually Lance joining our team was you connecting me with Lance, but Lance is highly technical in the way that he goes about devrel is very much like teaching, right? Like he writes a lot about memory and managing costs and these sorts of things, and he goes really deep. He builds evals. He like has a lot of really technical content that he puts together, and I really like that idea, like that idea of our, our devrel team being kind of like thought leaders for people- Yes... within what is a really nascent, really hard, really evolving space. Educators, thought leader, explainers of things. Yeah.”

Claim: Anthropic wants developer relations to provide technically deep education and thought leadership in the emerging agent-development field.

“Um, and there's another version of this I really like called the master builder, uh, which is I will build the things with the platform to show you what's possible, and then you'll be like, "Oh, I didn't know that was possible- Mm... with, with the Claude platform." Yeah. And then you, you just go. Yeah. Uh, and this is a thing stolen from Lego. Um- Oh, nice. Yeah, because Lego, how do they sell Legos? They have master builders. Yeah. They build things with Legos and you're like, "I wanna do- Well, that's the real cool... I wanna do that. Yeah. [laughs] I wanna be like this person." And then they, they, they do that. Nice. And so- That's awesome.”

Claim: The speaker describes a “master builder” model of developer relations in which concrete demonstrations of a platform's capabilities inspire developers to build with it.

“Um, and like tho- that was the era when you like needed to have that sort of thing, and now, um, you know, it's much more about we have a skill, like the Claude API skill, and it's like really, really excellent at helping you either within Claude Code or any other coding agents, um, to build a really high quality integration. And now like these sorts of things like, "I'm gonna build a computer use integration, like how do I go about doing that?"”

Claim: Anthropic provides a Claude API skill that helps coding agents build high-quality integrations with the Claude API.

“Like very few people I, I think/hope are reaching for the big gnarly cookbook, and more so they're loading up the skill on a coding agent and working through that. Yeah. Or they're just putting it into a reference architecture. Yeah. It actually do- does much better. Like increasingly we're seeing that for a lot of the things that you might want to have different implementations on, uh, typically at the harness layer, but there could be other variants.”

Claim: The speaker believes developers increasingly rely on coding-agent skills and reference architectures rather than large, complex implementation cookbooks.

“You just point Claude at it, and you say, "This is the reference architecture I want." Yeah. And then you build it. Fun fact, the Claude API skill that helps people build against the API is owned by our DevRel team.”

Claim: Anthropic's developer-relations team maintains the Claude API skill.

“This is the most, uh, high leverage form of DevRel is to like distill everything that is best practices that's gonna go into those agents. Yeah.”

Claim: The speaker views embedding developer best practices into agent-facing tools as a particularly high-leverage developer-relations activity.

“It's interesting because, uh, I think this is actually something we are trying to do more of with the platform team and with our team and, and folks who, you know, um, can kind of just get out there and like be part of the community and, and be speaking and be on X and be wherever they are, right? I think the problem for us is just like having time [laughs] and kind of knowing the right way to do that stuff at high quality.”

Claim: Anthropic is encouraging platform-team members to engage publicly with developers, while recognizing that limited time and uncertainty about quality are obstacles.

“So I th- I see the DevRel team in some ways as just being like the experts in doing this sort of stuff at quality. They can do it all day, every day, and they can help enable the rest of the team to think about how to do this with the small amount of time that they are able to spend on this sort of stuff to do it at high quality too.”

Claim: Anthropic's developer-relations team is intended to provide expertise and support that helps other employees engage publicly with developers effectively.

“Uh, and, and like- Absolutely... you know, you know. And you have like lots of... You have Tariq at Target. Yeah. Uh, y- you know, and Lydia- Why Chris and not at DevRel? You got Lydia through Bud, you know? Yeah. E- e- exactly. Yeah. But like he, he like functionally- He remains... he does the job... top of mind. He does the job. Mm-hmm. Um, and so like that, you want more of those- Yeah... 'cause people, people wanna hear from people in the product. Yeah. Not, not like who are like paid spo- paid shills- Mm-hmm... like paid, paid spokesperson. Yeah. And that's [laughs] Wh- which I get a lot of, and like that's fine. That's, that's a job. Somebody has to do it. Yeah. Like something like reasonably you can expect people who are, are working on engineering to not be good at it. Yeah. So like you should pay someone who specializes in it, like specialty is a, is a skill. But I, I think there is a back office/front office separation that, that can e- Hm... emerge. Yeah. That is really cool. Yeah.”

Claim: The speaker believes audiences prefer hearing from product builders over paid spokespeople, while recognizing that communications expertise is a legitimate specialized skill.

“What's something- Yeah... what's something people should try to push the bounds with new Opus 5.5? You guys have been using it for the longest. Yeah. I mean, it's really cost efficient. Um, it's less verbose.”

Claim: The speaker describes Opus 5.5 as cost-efficient and less verbose than its predecessor or alternatives.

“and, and people who were just like, "Amazing model. Thank you." Like one of my old bosses, um, messaged me, and he was like, "Opus 5.5 just saved our butts in an incident," right?”

Claim: The speaker reports that a former boss said Opus 5.5 helped resolve an incident and saved their organization.

“And so I think things that like maybe you wanted to experiment with something and like cost was something that was front of mind for you, right, and like token efficiency and these sorts of things, like that kind of might have held you back from like really like firing it off and like letting a model run. I think that's the... Those are the sorts of problems you should return to now with Opus 5.5 'cause I think you'll be able to get just like a lot further than you maybe otherwise would've been able to with, um, models that were less cost effective.”

Claim: The speaker recommends retrying cost-sensitive tasks with Opus 5.5 because its improved cost efficiency may enable more progress than earlier, less efficient models.

“I think you should try something with like a multi-agent setup. I think you will find it do very good. Any, any other hints, by the way? I, I, I mean, multi-agent's very broad. Uh, nope. Okay.”

Claim: The speaker expects Opus 5.5 to work well in multi-agent configurations and recommends experimenting with them.

“This is pretty good in a sense of like, basically, generally, you wanna have one lead, and you wanna like split up like a bunch of different stuff. And as long as you can have a task that you think could be generally parallelizable, and they share uniform context across it, I think you will get a lot of gains. Now, of course, it depends on the problem that you're pushing at, but like increasingly, this is a very superior way for us to handle a lot of these things, and especially if every sub-agent is, or whatever you wanna call these, like sub-leads, they are asynchronous, um, and they're not blocking of each other.”

Claim: A useful multi-agent design has one lead agent coordinating parallelizable subtasks that share a common context.

“And increasingly, I think a lot of this architecture stuff will fall into this direction. Like Claude will get smarter and smarter. It's going to learn how to figure ways to like split itself, clone itself, adjust itself to the architectures that make sense. You just need to prevent it...”

Claim: The speaker predicts that Claude will increasingly determine how to divide and adapt its work across appropriate architectures.

“And as long as you contain those two principles, honestly, I think people should experiment and probably try some really crazy stuff. And, uh, I wouldn't be surprised to see some like really interesting things on top of that, that maybe they felt like there was some intractable problem that was really annoying and they keep pushing for, that something like this could actually probably just like crush in maybe one shot or something.”

Claim: The speaker expects experimentation with unconventional multi-agent systems may solve some problems that have seemed intractable, potentially in a single run.