Pratik said the leaderboard combined benchmarks and datasets to create a holistic evaluation, since models may excel on particular datasets and data leakage may occur.

“Targeting certain domains for evaluation but what our goal was that let's take all of these benchmarks because it's possible that certain models are good at certain data sets there might be issues of leakage also so we wanted to create a as much possible holistic benchmark by taking on data sets slices on single turn multi turn”