The speaker says that when filtering cannot supply the needed training signal, it must be obtained elsewhere, for example from human-written reasoning, human-designed environments and objectives, or human data from deployment.

“just like filter until we see it right and so like at that point you have to get bits some other way either from humans like directly like asking them to like write out their reasoning or like by like creating environments where like humans decide like what environment should be created what the objectives of these environments are or like you know some kind of like training on like the human data that exists in deployment like you have to get the bits from somewhere”