The speaker’s team has struggled to train Qwen, Kimi, and Nemotron models using direct reinforcement learning, including GRPO, against the answers as given.

“We can frame some of these questions as multiple choice questions. Um, so yeah, we try training like Qwen models, Kimi models, Nemotron, and we really struggle if we just do direct RL, like GRPO training against, uh, you know, these answers as is.”