The speaker considers it risky to use a weaker judge, such as GPT-4o mini, to evaluate difficult outputs from a stronger model such as 3.7, because the judge may fail to recognize correct outputs.

“We try to use the best LLM available so that it should not be a situation where we are trying to evaluate 3.7 with GPT-4.0 meaning, right?”

“uh risky proposition because gpt4o mini can't find uh cannot evaluate the hard outputs that 3.7 might be doing correctly right um so and so for this v2 i am using the 3.7 because it is one of the top reigning models right now and i've seen really good performance from it so we essentially swap these models as much as”