The speaker suggests that a model may detect hallucinations earlier in its processing than it generates text, but that this check can occur too late to prevent the model from stating an incorrect fact.

“And it's quite possible this is also true for like hallucination and fact checking.”

“So it might be that like it takes the whole model somehow or the sort of generation step takes the whole model.”

“But it's it's like the checking happens earlier in the model.”

“So then when you put the incorrect fact through the model,”

“it's like, oh, yeah, that is a hallucination. But at that point, it's already said it.”

“It's too late.”