descriptive claim
The evaluation distinguishes a model's ability to use the Factorio API from its ability to set objectives for itself.
Alessio Fanelli · ⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins · 27 Apr 2025
In their words
The transcript segments this claim was taken from
“And so you have these two, let's say, abilities at play there of one.”
“Is the model actually good at using the API and working in Factorio?”
“And two,”
“what objectives does the model set for itself? So for instance, one of the interesting things we saw was that DeepSeek was quite decent at lab play.”9:45Report a misattribution
0.90how strongly it was put, not whether it is true
0.75importance to the conversation
0.94confidence the claim was read correctly