descriptive claim
Preliminary results suggest reasoning models perform worse than expected in this evaluation setting.
Jack Hopkins · ⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins · 27 Apr 2025
In their words
The transcript segments this claim was taken from
“One thing we have found in preliminary results is that the reasoning models don't seem to do as well as you'd expect in this setting.”12:12Report a misattribution
0.65how strongly it was put, not whether it is true
0.80importance to the conversation
0.98confidence the claim was read correctly