A diffusion-based language model might eventually be significantly more intelligent than an autoregressive model.

“uh we still use a lot of the public data sets that people have created and evals and benchmarks so you know we're a startup we try to be scrappy we try to use existing things as much as possible and kind of like focus on the on the the aspects where we can have the highest impact and then where we can be the most differentiated and right now it's speed in the future who knows like it's possible that a diffusion based language model will be”

“be maybe significantly more intelligent than an autoregressive one like nobody knows that that's why i think this is very exciting because we're developing these really powerful ai systems but it's all very fresh it's all very new uh i doubt we've discovered the best way of building these systems there's got to be alternative there's got to be other ways of building these models and and eventually yeah efficiency will be very important like if you think about the ai factor”