The speaker reports that his LLM-as-judge experiments for a hallucination index had correlation problems two years ago, improved substantially last year, and are very strong this year.

“just go back to annotation but I feel like it has started working a lot one year back because we I was running this experiments like we were working with hallucination index two years back where we were using our own similar LM as judge methodology and we had certain issues and there was some correlation issues at the time but last year it got pretty strong and this year I feel it's so strong I have LM as judges”