The speaker considers bug-finding fairly, but not arbitrarily, verifiable: demonstrating a bug may sometimes require a moderate-scale experiment on full distributed infrastructure.

“tasks to train AIs on because most of these bugs we're talking about can probably be demonstrated without that much compute and probably you get pretty good transfer from pointing out other types of bugs at smaller scale and so then you can RL AIs that like look at this overall complicated training situation and point out cases where there's like an important bug and then fix that and I think that like this is not like a this is like a pretty verifiable task it's not it's not arbitrarily verifiable because maybe often to demonstrate the bug you might need to do like”