The speaker identifies two speculative architectural possibilities: using diffusion rather than autoregressive generation, or analyzing model activations to produce weights.

“the other thing that's in the debate, obviously nobody knows because Jeff doesn't talk about it,”

“but the two speculations are,”

“one,”

“maybe diffusion model instead of autoregressive.”

“of but you you are able to achieve the parallel uh generation in”

“in in your way and then the other one is some mechinterp type thing that that you're like analysing the activations and then just outputting”

“weights which”

“like you guys have all done the research on this people have speculated”