The speaker characterizes robot rollouts scored by people as good, bad, or somewhat good as a robotics analogue of reinforcement learning from human feedback, with the resulting experience usable to train the model.

“see how the model behaves you figure out what it's not good at then you can like collect a bit more data then iterate right i think of this as like the robotics version of like reinforcement learning from human feedback right the robot rolls out you can score it good bad somewhat good and then that data the robot's only experience can be used to train the model and you can also just collect more data yourself to train a model right eventually you”