The speaker says it remains unresolved whether taste learned from shorter-horizon experiences generalizes to very long-horizon tasks.

“develop taste quite quickly from like a relatively short succession of like small things and so like theoretically it's you know possible to develop it like that the ai obviously will have vastly more experience in which to develop taste like metal on it and there's like how well does that generalize to like really long horizon things is i think the question which i think is really unsolved at this point like we don't know”