The speaker says rumors about lab training runs suggest significant difficulties in increasing pre-training scale.

“every single layer of this has been much harder than the strongest optimist expected it seems like there have been significant difficulties in increasing the pre-training size at least from rumors about field training runs or underwhelming training runs at labs it seems like building up these arlen i'm like total outside view i know nothing about the actual engineering involved there but just from an outside view it seems like building up the o1 like rl clearly took”