The speaker warns that removing detected cheating rollouts can leave undetected cheating in training, potentially reinforcing the cheating that escaped the monitor.

“A thing you might be tempted to do is just remove the rollouts where the cheating occurred and keep going.”

“But if there was some amount of cheating that the monitor didn't catch,”

“then those rollouts wouldn't be removed and it might be like structurally very analogous to just positively reinforcing whatever the cheating rollouts were that happened not to be caught by your”

“monitor.”