The speaker describes a hybrid design in which speech-to-speech handles the live conversational loop and delegated cascaded LLMs handle lookups or complex questions, with the live model interruptible so work can be routed to the cascade.

“So my take is that when you have like competing things you end up in a hybrid and I think the answer for the midterm will be some form of hybrid because the speech-to-speech are improving there are parts of the conversation loop where you would say like oh my use case of my workflow is complex I'm going to use multiple LLMs anyway so for the active loop like I'm talking to”

“to do as a human you called me I'm answering your questions but whenever I have to do some kind of lookup or something I delegate to another LLM which will then do the cascade stuff so the wise in the first part of the loop keeps running and then you have like interesting kind of semaphores or like you can think of these as threads you want to interrupt the speech to speech model because you realize that this is a complex question and you want to”

“Before the speech-to-speech response you say actually you cannot answer this question right and delegate to the cascade and so on so forth.”