Pratik said models that rank highly on general or other specific evaluations may rank differently on agentic tasks.

“100% yeah so we have been hearing from our customers that they are now moving towards agents and it become very clear to us that last year we did rag evaluation because that's what we are discussing more with the customers but we quickly understood that this is going to be a year of agents and what we have understood from our previous experiments is that it's not necessary that what you see as top”

“stop models in let's say Alemarina or other specific evaluation or general evaluations they might not be also the same ranking for other task like agentic task so that was our goal very fundamentally speaking that we want to get very specific insights apart from just performance and cost was the other thing that comes up a lot with enterprise customers and”