Discrete diffusion can be problematic when sampling from the density at every point; the speaker thinks language models are probably fine, but says the method requires more care.

“And that immediately allows you to do things like image generation, because I I explained just now with tokens to to make the comparison uh sensible, but ultimately you start up with something and you end up with something. In fact, discrete in this case is the more difficult part, because of discrete diffusion you you end up with some kind of dicey things with kind of uh I I guess language models are probably fine, but you get into dicey things with basically if you actually sample from the density at every single point.”

“It's there possible, but um you have to be a little bit more careful. But the nice part is that once you have kind of this this general recipe of decomposing your uh your your structure, then you can apply to anything. And this is what makes diffusion models or flow matchers which is like if you read diffusion model and you look into the into it, oftentimes it's flow matching because it tends to be a little bit more stable.”