Some models can improve substantially from as few as 100 data points when the data is pristine, not reward-hackable, long-horizon, and task-shaped, and may generalize from that data.

“some models can do very good hill climbing on just hundred data points if they are pristine data points. They are, they are not reward hackable, they are long horizon, they're exactly the shape of the, the task that they are trying to do, then small amount of data can work really well and the model can generalize from there.”