The speaker says weaker, older models may need examples of good and bad behavior in their prompts, and those examples can be removed later.

“At some level, it's also how the model was, right? Uh, weaker, older models, you need to give them examples of, "This is good, this is bad," and then obviously you cut that away at some point, right?”