Anthropic reports that its Rust Messages API improved time to first token by approximately 10–20%, with greater improvement on longer workloads.

“And then latency, I think we saw somewhere between, like, 10 and 20%, um, improvement on, like, time to first token specifically. So a lot of just really cool improvements, uh, across the board.”

“It's higher on, like, uh, longer running, like, workloads, but- So it'll, it'll roll out- Yeah... um, soon. Yeah. But it'll get faster. Do you fight the, the urge... You know, I've been involved in some major rewrites.”