In adversarial co-adaptation between language models, the attacked model can train against successful attacks to resist unsafe outputs, prompting the attacker to try new approaches when its attacks stop working.

“right they're like one makes a better attack then the first one inoculates itself somehow like uses that as training data makes it so it's harder to say something unsafe based on that and then as the attack stops working the attacker now tries a different angle right and that's why it's not just red teaming but they're called sort of But rainbow”

“teaming”