The team describes failure cases in general terms in the prompt and iterates until it is confident the LLM judge meets the required performance.

“Again, defining the failure cases in a general way to the prompt of the LLM is just and see if it's following it or not and that iteration goes on for multiple times till we are sure that we are getting the performance that we need.”