Alex Zhang thinks GPU-kernel programming has become widespread and may be nearly saturated as a field.

“because he was applying for faculty i mean he is faculty there now but i listened to his talk on flash attention in like 2023 and i was like wow this is like the coolest thing ever um and i was like this is like this is what everyone should be working on i guess like vlm and stuff had come out too and it was like oh you know we should be writing kernels but now it's like you know everyone writes kernels like everyone it's it's i think it's actually almost saturated in some sense uh as a field”