The speaker says that in an RLM, sufficiently offloading context and having the model write code over it can reveal that solutions to many tasks are similar when the model learns how to solve them during training.

“one of the things that we see with an RLM is that if you sufficiently offload context and ask the model to write code over that context,”

“you get this really, really weird, but useful property, which is that when the model recognizes during training how to solve a task,”

“it turns out that the solution to many tasks is”