⚡️Anthropic vs Cognition on Multi-Agents: A Breakdown with Dylan Davis

“Yeah, so I'm the founder of Gradient Labs.”

Claim: Dylan Davis is the founder of Gradient Labs.

“It's a company that helps other organizations, usually mid-sized companies, implement AI internally to help automate their processes, saving time, money, and sometimes generating revenue.”

Claim: Gradient Labs helps organizations, usually mid-sized companies, implement AI internally to automate processes, save time and money, and sometimes generate revenue.

“but there's, I guess, some discussion around the fact that both of these blog posts and context for the viewers or listeners is anthropic and both cognition posted blog posts one after the other. I think it was like June 12th and June 13th.”

Claim: Anthropic and Cognition published blog posts one after the other, around June 12 and June 13.

“One basically pushed the fact that multi-agent architectures are beneficial and useful and the other one basically stated the opposite, saying that single agent architectures is more.”

Claim: The two blog posts presented contrasting views: one said multi-agent architectures are beneficial and useful, while the other said single-agent architectures are superior for some use cases.

“more superior for different use cases and if you actually read into both blog posts you realize that they're both right so it's not either or but yes and if you're uh if you're uh what is it like the stand-up comedy thing yeah there you go that's it”

Claim: Dylan Davis concluded that both posts were right: neither multi-agent nor single-agent architecture is universally preferable, and the appropriate choice depends on the use case.

“So for the Anthropik blog post, they focused mainly on their use case for deep research. So I'm sure if you're aware,”

Claim: Anthropic's blog post focused mainly on a deep-research use case.

“if you have access to Claude, I think it's pro and max and all the paid plans, they have access to this new deep research feature that was released a few weeks ago.”

Claim: At the time of the conversation, Claude's deep-research feature was available on Pro, Max, and other paid plans and had been released a few weeks earlier.

“And with this research feature, I tend to compare a lot of the models. So when I do deep research, I do Gemini, I do Perplexity, I do OpenAI, I do all of them when I ask different questions.”

Claim: Dylan Davis compares deep-research offerings from Gemini, Perplexity, and OpenAI by asking them different questions.

“And time and time again over the last couple of weeks, I found that Claude has by far outperformed the others.”

Claim: Dylan Davis found that, over the preceding couple of weeks, Claude's deep-research system had by far outperformed the alternatives he tested.

“I guess the definition of good for me right now is not just length, but also the number of sources and diversity of response.”

Claim: Dylan Davis evaluates deep-research quality by more than response length; he also considers the number of sources and response diversity.

“My guess is part of the reason it's so highly effective is the way they've structured their agent.”

Claim: Dylan Davis suspects that part of Claude's effectiveness comes from Anthropic's agent architecture.

“So they took a multi-agent approach with this feature.”

Claim: Anthropic's deep-research feature uses a multi-agent approach.

“So this question is, what are all the companies in the United States working on AI agents in 2025? Make a list of at least 100. For each company, include the name, website, product description of what they do, the type of agents they build, and their vertical industry.”

Claim: The example deep-research prompt requests at least 100 U.S. companies working on AI agents in 2025 and asks for each company's name, website, product description, agent type, and industry.

“I would say the thing that's happening here that's useful for this specific structure for multi-agents is that it's a broad question. They're asking about a variety of things and they're asking for multiple perspectives on that specific thing.”

Claim: Dylan Davis says multi-agent architecture is useful for broad questions that cover varied subjects and require multiple perspectives.

“So what happens is they send the question over to their agent. agent architecture where it's where they have a lead architect so lead agent at the top that's orchestrating everything and they'll then send off this question broken into sub questions so send over the question they create a plan they create a bunch of sub questions and they send off these requests to multi multiple agents and they do it in parallel and that's kind of the unique thing here that i found it's and maybe so you haven't answered this or maybe you have insight is i've”

Claim: In Anthropic's described deep-research architecture, a lead agent plans the work, decomposes the question into subquestions, and sends those requests to multiple agents in parallel.

“I don't necessarily know, I can't confirm this is happening, but I know in their documentation it says that it is, where they'll send off a request as the lead agent to multiple subagents to then do the research.”

Claim: Dylan Davis could not independently confirm that Anthropic's lead agent sends research requests to multiple subagents, but said Anthropic's documentation describes this behavior.

“Yeah, well, because I'm a skeptic, all right.”

Claim: Dylan Davis describes himself as a skeptic.

“each one of these subagents have the 200k context window. And that means that all of them are going to have tons and tons of insights and research for that sub question.”

Claim: Dylan Davis says each subagent's research yields many insights relevant to its assigned subquestion.

“They'll synthesize that once they've synthesized the responses, they'll feed that back through the lead agent and the lead agent then will put a variety of insights that are relevant for answering the question in its memory. it'll then consolidate the memory insights and then create its report back to the user.”

Claim: In the described multi-agent workflow, the lead agent receives synthesized subagent responses, stores relevant insights in memory, consolidates them, and produces a report for the user.

“They'll synthesize that once they've synthesized the responses, they'll feed that back through the lead agent and the lead agent then will put a variety of insights that are relevant for answering the question in its memory. After that's done, it'll then consolidate the memory insights and then create its report back to the user.”

Claim: In the described workflow, subagents synthesize their responses and return them to the lead agent, which stores relevant insights in memory, consolidates them, and produces a report for the user.

“Another important caveat here is they have another citations sub-agent that's working as the sub-agents are responding to validate that the citation and the piece that is sent back is connected to the source that it referred.”

Claim: In the described workflow, a citations subagent checks during subagent responses that citations are connected to their referenced sources.

“Another important caveat here is they have another citations sub-agent that's working as the sub-agents are responding to validate that the citation and the piece that is sent back is connected to the source that it referred.”

Claim: In the described workflow, a citations subagent checks that citations and returned material are linked to their cited sources while the other subagents respond.

“And it is more detailed, but it's also more overwhelming. it is definitely more complex in the background.”

Claim: The multi-agent structure is more complex in the background than a simplified diagram suggests.

“So after we have a good understanding of multi-agent, I wanted to quickly show what a single agent architecture looks like. And this is a chart from Cognition's blog that walks through a basic flow or basic architecture of what a single agent could look like.”

Claim: A Cognition blog chart presents a basic architecture for a single AI agent.

“So in this case, we have the task at the top where it's getting an input from a user. So the user's input comes in and we go through the same process. So we're going to have the large model at the beginning. it's going to create a plan, break down that plan into subtasks, subquestions, etc.”

Claim: In the described single-agent architecture, a large model receives user input, creates a plan, and breaks the plan into subtasks or subquestions.

“The difference here is instead of doing it in parallel with subagents, they're going to do it in sequential order. it'll first research the first question, and it'll then consolidate the insights from that and it'll go to the next question and it'll go to the next question and it'll go to the next question. And.”

Claim: In the described single-agent architecture, the agent researches questions sequentially and consolidates each question's insights before proceeding to the next.

“This order works for some use cases, but when it comes to research, there are some downfalls.”

Claim: Sequential processing works for some use cases but has drawbacks when used for research.

“And one of the downfalls is the fact that the context window is going to get bloated pretty quickly, which means that the variety of insights and diversity of insights is limited based off of what it can pull within those iterations when it runs literally.”

Claim: In sequential AI research, the context window can become bloated quickly, limiting the variety and diversity of insights available across iterations.

“And that's kind of one of the benefits of the multi-agent structure, where you can have a multitude of agents doing research from different perspectives, synthesize that without loading the context window too soon.”

Claim: Multi-agent research can gather perspectives from multiple agents and synthesize their findings without filling the context window too early.

“So in the blog post that Anthropik posted, they ran some tests and they noticed that the multi-agent structure outperforms a single agent structure by 80%. based off a different variety of variables they measured.”

Claim: Anthropic's blog reported that tests found a multi-agent structure outperformed a single-agent structure by 80% across the measured variables.

“So you can see here that the use of an agent for research, if we started with one, so this is one over here, that's how many imagine tokens it would take. So based off of a basic conversation, a single agent architecture for research is around 4x the number of tokens needed to achieve a research output.”

Claim: For a basic conversation, a single-agent research architecture uses about four times the tokens needed to produce a research output.

“When you use multi-agent architectures, it's actually 15x the number of tokens.”

Claim: The described multi-agent architectures use 15 times the baseline number of tokens.

“So this is one thing they've called out is that the token usage is a lot higher, but the trade-off they consider is worth it because the quality of the response is much higher as well.”

Claim: The reported trade-off for multi-agent architectures is higher token usage in exchange for much higher response quality, which the source considers worthwhile.

“So yeah, so this is basically calling out the evals and the importance of structuring these. So there's five dimensions they looked at. So first one's factual accuracy. So was the claim, did the claim match the source? Next one is citation accuracy, which is a kind of a call out to the subagent that I mentioned over here, where we're checking the source and ensuring that the source supports the claim based off of what's being referenced. Next is completeness. So are all aspects of the question being addressed? Next is source quality. So is it a good source or bad source? primary source was high quality and the secondary source was less high quality and then the last one here is tool efficiency so did they call the appropriate tools at the right time in the right way so they measured all five of these as a as a way to evaluate the output from the model yeah”

Claim: The report evaluates model output across five dimensions: factual accuracy, citation accuracy, completeness, source quality, and tool efficiency.

“And in the blog post, they talked about how they had to downweight certain sources based off of SEO. So there's some companies that are gaming the fact that if they put a lot of words in their search terms, they might get picked up by some of these AI models.”

Claim: The blog post said some sources were downweighted for SEO because companies can manipulate search terms to increase the chance that AI models select their websites.

“So they had to downweight some of those marketing websites and replace them with higher quality blogs and forums and things like that to then ensure that the primary. primary source was high quality and the secondary source was less high quality and then the last one here is tool efficiency so did they call the appropriate tools at the right time in the right way so they measured all five of these as a as a way to evaluate the output from the model yeah”

Claim: The blog post said some marketing websites were downweighted and replaced with higher-quality blogs and forums to improve source quality, distinguishing high-quality primary sources from lower-quality secondary sources.

“I think it's important to note that it's an LLM judge. We just got a paragraph on it, right? They didn't actually open source anything about how they do this.”

Claim: The interviewer says the evaluation uses an LLM judge and that the authors did not open-source details of their evaluation method.

“I think it's important to note that it's an LLM judge. right? They didn't actually open source anything about how they do this.”

Claim: The system uses an LLM judge, but its authors did not open-source details about how the judge works.

“One of the things they mentioned in there around the LLM as a judge is they initially started with five LLM as judges. So each one of these points had their own LLM as a judge.”

Claim: The system initially used five LLM judges, assigning one judge to each point or criterion.

“They tested the ability and accuracy of that LLM as judge collective to judge, and it actually didn't perform as well as one. So they replaced all those judges with one judge, and the one judge actually performed out the collective.”

Claim: In their tests, a collective of LLM judges judged less accurately than a single judge; they replaced the collective with one judge, which performed better.

“right? Instead of calling the same thing with the same context five times with slightly different problems, you can just like ask for all of the scores. It's like kind of an efficiency thing, but it's actually comforting to know that it's more consistent and more aligned.”

Claim: The speaker views using one judge to assess all criteria as more efficient and finds its greater consistency and alignment reassuring.

“I think the other thing that you want to know is I think once the judge looks at all the criteria, they can separate out more instead of. overlapping discourse,”

Claim: The speaker thinks a judge that considers all criteria can distinguish them more clearly instead of allowing their discourse to overlap.

“I mean, coding is like the current hot thing. Everyone's building background agents and sub agents and parallel agents and all that.”

Claim: Coding is currently a prominent use case for building background agents, subagents, and parallel agents.

“So that's the obvious one where I think for a while, a lot of coding agents have been exploring the sort of orchestrator agent versus like, you know, the worker bee agents for a long time.”

Claim: Coding agents have explored orchestrator-and-worker-agent architectures for a long time.

“offer write access to the world so deep research is read only so it's the safest to just spam out as many as you want because it will never have side effects on the world”

Claim: Because deep-research agents are read-only and cannot affect the world, the speaker considers it safest to run many of them.

“coding you have version control so if anything goes who's up you can sort of rewind with git and all that i think the hardest will obviously be some agents that actually have impacts on the real world that you cannot”

Claim: The speaker expects agents with real-world effects that cannot be undone to be the hardest to manage.

“coding you have version control so if anything goes who's up you can sort of rewind with git and all that i think the hardest will obviously be some agents that actually have impacts on the real world that you cannot”

Claim: Version control in coding allows a developer to rewind changes with Git if something goes wrong.

“Yeah, context engineering. It's the hot new thing.”

Claim: Context engineering is currently a popular topic of discussion.

“Everyone's talking about it only because Andre talked about it. But also like it was already trending before Andre talks about it.”

Claim: The speaker says discussion of context engineering increased after Andre talked about it, but the topic was already trending beforehand.

“When people think about prompt engineering, they're like, you know, you are a, you know, very, very good software engineer. And if you get a wrong answer, your grandma's going to die. You know, that's prompt engineering.”

Claim: The speaker characterizes prompt engineering as giving a model a role instruction, such as being a very good software engineer, and emphasizing severe consequences for a wrong answer.

“But context here is it's so much more.”

Claim: The speaker argues that context encompasses substantially more than prompt engineering.

“And so that's something they discuss in the blog as well, or at least hint at, is that the context for this multi-agent structure is distributed and not centralized.”

Claim: The blog describes the multi-agent structure's context as distributed rather than centralized.

“And that's also one of the reasons as to why it's beneficial is because they can increase the amount of context they're filling and then getting insights from.”

Claim: Distributed context in a multi-agent structure is beneficial because it allows agents to process more context and obtain insights from it.

“So Cognition's blog post was basically the exact opposite, stating that multi-agent architectures aren't necessarily that fruitful and you should probably go with a single agent if you're going to build something of any use.”

Claim: Cognition's blog argues that multi-agent architectures may not be fruitful and that a single agent is probably preferable for building something useful.

“And it's interesting because this just comes to really their perspective and bias on the world and their experience. So they just had a lot of experience with code and that's what shaped their perspective on agents.”

Claim: The speaker believes Cognition's experience, especially its extensive experience with code, shaped its perspective on AI agents.

“and Claude, on the other hand, probably has both experience with research and code, but they had a slightly different perspective on this.”

Claim: The speaker thinks Claude probably has experience in both research and coding and has a somewhat different perspective on multi-agent systems.

“So first things first is the context is centralized, like I mentioned. So that means the context is going to remain with a single model or a subset of models that kind of get passed off to each other.”

Claim: In a centralized-context architecture, context stays with one model or a subset of models that pass it between one another.

“And in this case, let's say the task is to create a Flappy Bird game. That's what we're trying to achieve here. the orchestrator, is going to break that down to subtasks, and we have two subtasks. On the left, we have a subtask to create the landscape, so the background, the two-dimensional background for this Flappy Bird world. we have creating the character itself, so the bird, and also the physics associated to the bird as kind of how it moves. So on the left hand, we have our sub-agent that's working on the background,”

Claim: In the speaker's Flappy Bird example, an orchestrator agent divides the game-building task into subtasks for the background and for the bird character and its movement physics.

“and it goes off the rails and decides to create something related to Mario. Which is cool, but not what we want. we have Flappy Bird. So with Flappy Bird, we're building out the icon and the physics that works all good.”

Claim: In the example, the background subagent produces Mario-related scenery instead of the intended background, while the other subagent correctly builds the Flappy Bird character and physics.

“But then we merge these. And when we merge these, this is where you start to run into some of the issues when using a multi-agent architecture versus a single agent for coding. And the main issue is that these are dependent upon each other.”

Claim: The speaker argues that multi-agent coding can run into problems when subtasks are interdependent and strongly coupled.

“They're strongly coupled in the sense that when I take an action, that action then impacts all the other subagents if it's taken incorrectly.”

Claim: In a strongly coupled multi-agent task, an incorrect action by one agent can affect all the other subagents.

“you'll see that the final result looks amazing, but is incorrect. Where we have Flappy Bird inside of Mario World and it's not a win because Flappy Bird doesn't get to achieve their goals and nobody wants to not achieve their goals,”

Claim: In the example, merging the agents' work produces an impressive-looking but incorrect result: Flappy Bird appears inside Mario World, failing the intended task.

“It's right here. Suppose your task is build a Flappy Bird clone and then it says the background that looks like Super Mario Brothers.”

Claim: The source example asks for a Flappy Bird clone with a background resembling Super Mario Brothers.

“So that's kind of the reason as to why multi-agents won't always work for coding because of dependencies.”

Claim: Multi-agent approaches are not always effective for coding tasks because coding tasks can have dependencies.

“So instead of doing that, they do a single agent approach.”

Claim: The approach described for coding uses a single agent rather than multiple agents.

“So we have the task at the top coming in and then we have our main lead agent that's going to break down and create the plan and all that stuff, but then it's going to go work in sequential order with the context that's been passed.”

Claim: In the described single-agent workflow, a lead agent decomposes a task and creates a plan, then proceeds sequentially with context passed along.

“The one thing that they've changed here is that after each action is taken or subset of actions are taken, we're going to... condense down the main moments and decisions and actions taken from the previous conversation or context and pass it off to the next agent.”

Claim: In the described workflow, after each action or group of actions, the system summarizes key moments, decisions, and actions from the preceding context and passes that summary to the next agent.

“And the hard part here that they mentioned in the blog post is being able to decide which actions, decisions, and moments matter most to be passed on. So that's kind of one of the trickiest parts of this.”

Claim: A key challenge in the described agent workflow is selecting which prior actions, decisions, and moments are important enough to pass to the next agent.

“And by doing this approach with single agents, you're likely going to achieve more success because you have all the context from previous agents the reason it's beneficial is that all the actions taken previously are baked into that next agent's context so it doesn't necessarily contradict or conflict with what was done before and that is cognition yeah”

Claim: The speaker expects the described sequential single-agent approach to be more successful because each agent receives prior context and actions, which helps avoid contradicting or conflicting with earlier work.

“it gives you an idea of when to make that decision and it's not just i mean right now the examples are research and code but there are some abstract questions one can ask themselves so the the first is can can you break the task into independent parts where they're not reliant upon each other so like that's one abstraction away”

Claim: A criterion for choosing a multi-agent approach is whether the task can be divided into independent parts that do not depend on each other.

“And in that case, again, research works.”

Claim: The speaker presents research as a task that can benefit from multiple agents' perspectives.

“And then the last one here is the token usage. So can you 15x the cost? And is the trade-off worth it?”

Claim: When considering multiple agents, evaluate whether a 15-fold increase in cost is acceptable given the benefits.

“And then for single agents is kind of the opposite. So does everything depend upon each other? If one thing breaks, does that impact everything else?”

Claim: A reason to favor a single-agent approach is that task components are interdependent, so one failure could affect the entire workflow.

“Does it need to be reliable every single time based off of what the context being passed?”

Claim: When choosing a single-agent approach, consider whether the workflow must be reliable every time given the context passed between steps.

“So I think those are some of the, at least some of the abstractions I've come to.”

Claim: The speaker describes task independence, value from multiple perspectives, token cost, interdependence, and reliability as some of his general criteria for choosing between agent approaches.

“yeah so it's actually it's worked pretty well for me a few times i mean i can i don't know if you want i can show you a few other charts and other things that i've done but”

Claim: The speaker reports that using Claude to create visual artifacts has worked well for him on several occasions.

“yes i always uh i always i'm a big fan of visuals like obviously that's kind of why i do youtube and teach”

Claim: The speaker says his enthusiasm for visuals is one reason he makes YouTube videos and teaches.

“yes i mean like that's that's one big one is i'll come up with an idea of something i want to talk about and then i'll think about like what's a good visual that represents the primary point i'm trying to make”

Claim: The speaker develops visuals by first identifying an idea to discuss and then choosing a visual that represents its primary point.

“the improved capability of a model if it can reason and think over time so this is a visual that cloud created Wow okay”

Claim: A visualization illustrates that a model’s capabilities improve when it can reason and think over time.

“and then there's another one over here that it just had here we go so this was another one for it's not as great it's not as beautiful but this is for for codex so it's basically talking about the spectrum of human intervention and autonomy”

Claim: A visual for Codex presents a spectrum ranging from greater human intervention to greater autonomy.

“And how basically as you can lay these out and you see that there's less human intervention for some of these tools and more so.”

Claim: The tools shown differ in how much human intervention they require; some require less intervention than others.

“You shouldn't choose it because everybody else is talking about it, kind of the whole like contradictory Devin thing of saying single agent actually works as you should define the problem you're trying to solve and then figure out what agent architecture is most suitable for that problem instead of just choosing an architecture.”

Claim: Agent architecture should be selected based on the problem it needs to solve, not because the architecture is popular or widely discussed.

“I think the only comment I'll make up if you kind of scroll up here is that generally everyone I talk to is just disregarded cost.”

Claim: The speaker observes that most people he talks to generally disregard cost.

“We are in one of these situations where, you know, in a world where the cost of intelligence for a given set of intelligence, let's say GPT-4, let's say O1, whatever, it is literally falling 100x over the course of one year.”

Claim: The cost of obtaining a given level of AI intelligence, such as GPT-4- or O1-level capability, is falling by a factor of 100 over one year.

“So you should actually build your products ahead of where costs are so that by the time they are popular, actually, you're frontier.”

Claim: Products should be built ahead of current AI costs so that they are at the frontier by the time they become popular.

“So it's actually, I think the efficiency thing trips up a lot of people in terms of being cost conscious and wasting a lot of time on that when actually the most cost unconscious people tend to win.”

Claim: In AI product development, excessive focus on efficiency can lead to wasted time on cost-consciousness, and the least cost-conscious people tend to win.

“should also the ML mindset may not adapt well to like if you if you came from that world you're very much about like let's do the smallest model that that hits our acceptance criteria because we want to serve it at very large scale”

Claim: A traditional machine-learning operations mindset favors the smallest model that meets acceptance criteria when the model must be served at very large scale.

“Here it's the other way around where the probably more you spend on intelligence, the more ahead in the future you are.”

Claim: For AI businesses in this context, spending more on intelligence can put a company further ahead of future cost and capability conditions.

“You can grab a lot of users, you get more funding to extend your runway and hopefully at some point you'll have a sustainable business.”

Claim: Investing more in AI intelligence can attract users and funding to extend a company’s runway, with the hope of eventually achieving a sustainable business.

“We're like Uber and Lyft competed against each other and Uber and they just basically burned a bunch of money.”

Claim: Uber and Lyft burned substantial amounts of money while competing with each other.

“so LinkedIn is probably the best place. I don't frequent X.”

Claim: The speaker says LinkedIn is probably the best way to contact him, and he does not use X frequently.

“Like I said, it's a Gradient Labs as a company. We do AI consulting for internal implementation of automation.”

Claim: Gradient Labs provides AI consulting to help organizations implement automation internally.

“And also I have a free 30-day email sequence. So if you're interested in getting more insights on AI that's kind of no BS fluff to apply to your own work and your own business,”

Claim: The speaker offers a free 30-day email sequence of practical AI insights intended for use in subscribers’ work and businesses.