The speaker says pre-training compute-efficiency gains can be quantified by measuring whether a change reaches the same pre-training loss using less compute.

“And in free training,”

“you do it by,”

“you have your pre-training loss,”

“and then you measure your compute efficiency gains against,”

“you made some changes, now you have a new pre-training loss,”

“and it should be that you got to the same point with less.”