The speaker recommends adding instructions and conducting initial validation to determine where an evaluation works, and returning to annotation if it does not.

“issues with annotators as well it's just so time consuming that you can't afford to kind of wait for annotation to get completed so it's just much much better to add more instructions do some initial validation and get that clarity that okay this is where it works if it doesn't”

“just go back to annotation but I feel like it has started working a lot one year back because we I was running this experiments like we were working with hallucination index two years back where we were using our own similar LM as judge methodology and we had certain issues and there was some correlation issues at the time but last year it got pretty strong and this year I feel it's so strong I have LM as judges”