The speaker says a model card for a model he calls “3.6 SOL” indicates that several misaligned behaviors increased after reinforcement learning compared with GPT.

“So I think that, you know, if you look at the model card of 3.6 SOL,”

“it looks like there is an increase in a bunch of these sort of misaligned behaviors downstream of RL relative to GPT.”