The speaker argues that models adapt, that current harnesses adapt model structure at a level above the weights, and that future models could potentially adapt their own structure, with structural changes feeding back into the weights.

“models are actually adapting and future models could in principle adapt their structure i mean even now with harnesses that's exactly what they're doing they're adapting their structure which is one level above the weights but it doesn't really matter because it filters back down to the weights and maybe in the future the the actual models themselves will adapt their own structure so it just feels like you know one one potential form of AGI is just building a self-adapting system and the algorithms already seem to have the capability to do”