Humans remain valuable for improving models, and the speaker expects human preferences to play a large role in giving models taste.

“And so humans are still th- really valuable in making these models better. And so we rely, we will rely a lot on, uh, human preference, um, to imbue things like taste.”