Coding evaluation combines verifiable rewards with human judgments of taste.

“Um, the models have gotten-- I would say coding is not fully solved, but like let's say we're eighty percent of the way there, and we're making progress on the, on the last twenty percent. Um, coding is a combination of verifiable rewards and human taste, right?”