Even if in-context experience makes AIs highly capable, those gains would be temporary and wasted unless the learned information can be incorporated into model weights.

“And even if after enough in-context experience the AIs can become like Henry Ford or Robert Einstein,”

“all that would be ephemeral and wasted if you couldn't get those learnings back into the weights.”