causal claim
In the team's tests, vision did not improve performance and sometimes worsened it by causing models to hallucinate nonexistent entities and make claims unrelated to the game state.
Alessio Fanelli · ⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins · 27 Apr 2025
In their words
The transcript segments this claim was taken from
“As you said, adding this niches didn't improve things, and it sometimes made it even worse because they started hallucinating entities that weren't there and started doing often,”
“went on tangents that weren't actually real in the game state.”
“Right now, at least, the vision actually wouldn't improve.”15:08Report a misattribution
0.90how strongly it was put, not whether it is true
0.80importance to the conversation
0.96confidence the claim was read correctly