Model routing is inherently difficult because task requirements and model capabilities are imperfectly known, producing errors even with strong classifiers.

“Like, with the router you, you go through, it's like, let's say you want to do a cost router, right, for the task complexity that you have. Makes sense. Uh, very hard to do.”

“Uh, you don't know everything upfront about your task, so you can't, like... Even with perfect, you just don't actually have perfect information.”

“Even if you could actually understand each model perfectly, which you don't, but even if you did, you also don't have perfect information about your task.”

“And so there's, like, a lot of error associated with your, your routing logic that you would have to create, whether no matter how good a classifier you produce.”

“And yeah, maybe it's, like, a super big brain model type of problem, but regardless, it's I think a very difficult one.”