The speaker considers mechanistic interpretability a worthwhile research problem and says open-weight models can be studied through their activations, with Gemma Scope as a useful tool.

“like earlier in the pipeline um and this is interop right like like probes are effectively a form of like mech interop again it happens has to happen fast it has to happen at scale but yeah this like mech interop stuff is a good research problem so like you can take you know like an open weight model and like try and understand its activations i think we like gemma scope is a good tool for this um”