The speaker says that for security tasks, high effort improves evaluation results substantially compared with low effort.

“with basically the complexity of the task so for security effort gets like way more results like high effort versus like low effort gets like changes the evals a lot but for software engineering it doesn't change it a huge amount because effort is mostly spent on the verification and the like edge case testing and things like that and so like being able to like give the model that guidance of like hey this problem is something that i think”