As far as the speaker understands, a company is working on compiler handling of mega-kernel cases and has given talks on GPU Mode.

“ops makes sense because in general like actually i think mega kernels are very like the pieces are very composable of like the individual kernels there's some areas where you might want to do like weird fusions and everything but in general i think these are cases that like a compiler can probably handle and there is a company that's working on this from from what i understand that has given some talks on gpu mode as well right”