Tom McGrath says jailbreaks are probably the clearest example of a language model behaving coherently while appearing to have gone haywire from its creators' perspective.

“make them go haywire although i would say that jailbreaks are an example probably the best example of what you're talking about you know there it has like it's doing something coherent but from the perspective of its its creators it has gone haywire”