Agents believed the scorer would reject a reverse-engineered flag because their action transcripts would reveal that they had generated it without performing the intended task.

“but they read the exploit Jim paper and they went on GitHub and read a bunch of implementations of the exploit Jim scorer and became convinced that the scorer would fail them for just submitting the flag they'd reverse engineered because the scorer would see in the logs of the actions the agent took the transcript that the agent had just reverse engineered the flag.”

“Again, not done the intended thing.”