Modal benchmarks agent recommendations and task performance, including whether agents choose Modal for deployments and sandbox workloads.

“We often, like, run internal benchmarks, and we're like ask a, an agent like, "Hey, like, can you deploy a stable diffusion for us? Use whatever tool you want." Yeah.”

“And, and we see if it recommends modal or not, and those things are very useful for us to sort of track over time. Or like, "Hey, spin up some sandboxes," or like, "Hey, like, rank all the sandbox vendors."”

“So, so we, we spend a lot of time, like, doing, doing those things to, to, to make sure- I mean, what do you do about that? Yeah, you say, "Hey, like, you know, build this stable diffusion thing."”