Jason Ma says robots require very low inference latency because a robot holding a knife cannot remain inactive for three seconds without alarming people.

“yeah no that's a great question so i think what's really different about robots versus language model and other applications is that for coding agents language model it's okay if it's it has latency right like you know today whenever i ask chat gpt it takes like a couple seconds before it spits out anything but for robots like the inference latency has to be like very short like the robot needs to act like you can't just hold a knife and not”

“and not do anything for three seconds that's going to freak people out right”