One possible mechanistic explanation for why a model may generate a hallucination yet recognize it when asked is that its internal operations occur in a particular order.

“it will sometimes be able to pick it up um literally the same model that just hallucinated will be like oh if you ask it it'll be like oh that is a hallucination um and it might be that like this is actually a very hard thing to supervise you know if you try and supervise like if you this is like a hard and expensive thing to just put into a into training supervision um an interesting kind of mechanistic hypothesis for this um”

“is to do with the ordering of operations inside the model so we've seen this in in in arithmetic that like um sometimes like things could things things have to happen in certain orders like layer layer nine has to occur before layer 10 and so on um and you know you have different modules that if they like”