Andrej Karpathy suggests that a model's experience could be processed through synthetic-data generation and distilled into its weights, potentially with a person-specific neural network.

“basically doing some kind of a synthetic data generation process and distilling it back into the weights and maybe having a specific neural net per person,”

“maybe it's a Lora, it's not a full weight neural network that's...”

“So it's just small, some of the small sparse”

“subset of the weights are changed.”