AI software R&D includes work on libraries, kernels, reinforcement-learning environments, and optimizers, not only the invention of new transformer architectures.

“And a subtle point that they'd often make here is software R&D in AI is not just alien type coming up with”

“new transformer like architectures to your point it actually is a lot of like you gotta like i mean i'm not an ai researcher but i assume there's like from the lowest level libraries to the kernels to making rl environments to finding the best optimizer to there's just like so much to do and in like parallel you can be doing all these things or finding optimizations across them and so you have two effects going back to this”