Jason Ma says current frontier models have strong vision capabilities that support real-to-simulation workflows and 3D reconstruction.

“that kind of thing was still very limited at the time because back then the model didn't really have good vision capability didn't have an innate 3d understanding right so it was basically just generating code and like using very limited feedback from text to try to come up with new simulation environments but today like the vision capability of these frontier models are crazy they can do like real to sim they're catting there's 3d reconstruction all of that”