The speaker recalls believing in OpenAI’s early days that minimizing log loss alone would not yield intelligence because important information made up too little of the loss and would be overwhelmed by noise.

“Yeah, in fact, I remember in the early open AI days having the intuition that actually just do like minimizing log loss wasn't going to get you to intelligence because like the important bits are accounting for.”

“There was such a small fraction of the loss that like it was going to be overwhelmed by noise.”