An AI examining the Kaplan scaling laws might have identified years earlier that intermediate checkpoints were used without accounting for annealing, potentially saving one or two years of progress.

“you know if you've got an ai to think about like the caplan scaling laws like an ai at this point would have noticed that like you know oh like they've just taken these intermediate checkpoints and didn't account for like the annealing and so like this is wrong and like that would have called that like years earlier we would have made like you know progress like like would have cut off a year or two of progress just from that like observation from an ai um and like again once the objective is well specified which is like lower pre-training loss or whatever”