Before publishing an environment for tasking, the team runs agent rollouts to check for reward hacking and has an expert review the results.

“You know, we'll run like a bunch of rollouts against it and try to like, uh, uh, investigate like are-- is the agent doing anything to reward hack the environment?”

“Um, and the expert will sort of like review those things before we publish an environment for tasking. Right? So that baseline is just all that has to happen before you can even start the like tasking part, right, in, in theory.”