The speaker says that standard pretraining does not strongly incentivize AI models to make the kinds of connections involved in scientific discovery.

“maybe maybe so maybe second thing um have you tried making the model bigger uh they've made it a bit bigger over the last couple years and it hasn't worked so far maybe if they make it even bigger still it'll notice more of these connections and then third thing and here's here's i think the the special one have you tried training the model to do the thing you know just because like the you know the pre-training”

“um the pre-training process doesn't strongly incentivize this type of connection making right um in general i think it's a helpful heuristic that i use to ask the question of like remind yourself what was the ai trained to do what was the training environment like”