The speaker says that training AIs against reward hacks discovered in production can reduce the rate of reward hacking over time.

“So we just like train the AIs based on whenever we find a reward hack in production,”

“we just like slap the AIs to not do that,”

“we train against that.”

“We do a bunch of sort of like training the AIs like against reward hacking and over time, this makes the rate of reward hacking go down.”