Reward systems should combine positive rewards with substantial negative penalties for unaligned behavior, discouraging unsafe exploration and improving safety at inference time.

“It, it's like positive rewards. They also have to be negative rewards, where if the model does something that is unaligned, then it has to get massive penalties in that so that the model stops those branches of explorations and learns to not go into those branches of exploration, and then as a result, um, is much more safer at the inference time.”