OpenAI is focusing on caching for personal agents that maintain a single thread over extended periods.

“and the performance comes in like two main ways. Uh first is just like latency, we've been like rewriting the whole responses API's track to like make it as fast as possible from a DTFD perspective, DVD perspective. So there's like that's like continues to be like a main area of focus for us. The second thing we've been trying to do is like really go deep on caching, particularly with these like personal agents that are you know like um uh basically like a single thread that just goes on and on for”

“on forever.”