The speaker thinks reinforcement learning probably has substantial discoveries remaining, which feel more like engineering discoveries than breakthroughs in new science.

“I don't see there hasn't been an asymptote of how much how much juice you can get out of better pre-training like we're not seeing a slowdown in compute efficiency gains you keep seeing stuff there's a lot a lot of headroom and reinforcement learning I think that that one is probably there's a lot of stuff to be kind of discovered there but it's again it feels more like a engineering discovery than”

“groundbreaking understanding of a new science or something like this.”