The speaker believes running the model over data with a sparse autoencoder is probably cheaper overall because it avoids generating tokens for every example.

“And in fact, this is probably like on net cheaper because you're not asking it to generate tokens for each example.”

“You're just, you know, you're just like you're just in the pre-fill regime. You're just pushing loads of data through and saying,”