When a pretrained model already knows an answer used in a reinforcement-learning task, it can appear to do the work without performing the required physics reasoning, calculations, or simulations.

“do types of programs that would be harder externally because if the pre-trained model has already memorized some of this data and then you try to do reinforcement learning on that,”

“if it already knows the answer and you construct reinforcement learning tasks that depend on that answer,”

“it can fake work.”

“so it doesn't actually have to do the hard physics reasoning. It doesn't have to actually...”