OpenAI API latency can spike, and users have no control over those latency spikes, according to the speaker.

“and then just more reliable I think I'm not sure if everyone faces this but like open AI API spike and then you have no control on those latencies with if you sort of have a self-hosted model you could scale up scale down based on your requirement so that's one thing we are seeing a lot with our customers.”