The speaker describes a possible waterfall architecture that streams to multiple destinations, including a small language model capable of fast inference.

“To your point where you can have, think about it as a waterfall,”

“you can actually stream to multiple places.”

“So what if you have streaming down to a small language model,”

“which can do inference in a very little time.”