Next-token prediction training does not incentivize models to discover the connections being discussed.

“are incentivized to find these connections next token prediction just isn't incentivizing them to do this but the rl required to do this would not be that or set up some sort of scaffolds i think actually google deep mind did do some similar like scaffold to make new discoveries and i didn't look into like how impressive the new discovery was they claim that some new discovery was made by an llm as a result um”