The speaker says the amount of intelligence obtained per training FLOP is increasing.

“that yeah i mean it depends on you know what metric it is but it roughly i do think that the speed is probably four or more times faster than researchers just doing it alone and that'll probably accelerate so the amount of intelligence you can extract”

“per training flop is is increasing so at some point that asymptote is okay and then also like the amount of revenue that each flop can generate also increases right the more you pack intelligence so going back to the question around how does this distribute across pre-training and reinforcement learning and to train beam which is a 500 billion parameter model total 23b active it was”