In a regime of recursive self-improvement, training methods, rewards, environments, and possibly the concepts used to understand them may change very rapidly.

“I mean, especially if you enter a regime of recursive self-improvement where the nature of training,”

“the rewards we're using,”

“the environments we're building,”

“maybe these concepts themselves will be changing very rapidly.”