Jason Ma

“Like if you're trying a world model that's purely based on videos, I think it will be a very powerful world model, but it may not be a sufficiently rich enough world model.”

Expects: The speaker thinks a video-only world model could be powerful, yet may lack sufficient richness for robotics. · no date given · not yet resolved

“So the reason I bring this up is because I think if vision tactile world model or vision tactile audio, whatever world model is the final thing that solves robotics, then I think robotics companies will be able to do that better than.”

Expects: The speaker believes robotics companies would outperform video-generation companies that pivot to robotics at developing the eventual solution, if that solution requires a vision-tactile or vision-tactile-audio world model. · no date given · not yet resolved

“embodied actor human but the issue with human data is that before your robot looks exactly like a human there's going to be some gap between human actions and the robot actions right so just by training a human data today it's not very likely you can just execute robot actions directly so you still need some amount of adaptation using robots own data right but the robots own data”

Expects: Training a robot only on human data is unlikely to enable it to directly execute robot actions. · no date given · not yet resolved

“right because today people have looked at egocentric data robot data you know those are scaling but i think since that static data will be a new dimension of scaling so going back to your question like yeah we have been exploring like using frontier models to generate synthetic environments for us on our data on our environments that we care about and surprisingly i think now is at a point where this technology is actually”

Expects: Jason Ma expects static data to add a new dimension to robotics scaling. · no date given · not yet resolved

“now it's at a tipping point where. this technology could really accelerate robotics and for us right we think about that as like a way to accelerate our internal progress right because we're building models we're building hardware so if a frontier model can give us more data that way like generate really tailored data for ourselves that's extremely useful for us and another thing we've been thinking about is instead of using these models to generate like”

Expects: Frontier-model technology may be at a tipping point and could substantially accelerate robotics. · no date given · not yet resolved

“the world action model I was telling you about because the model not only generates action, can also generate videos the model can also generate its own training data so that becomes a, you know, self-fulfilling like flywheel that I think will also be powerful”

Expects: Jason Ma believes that a model generating its own training data could create a powerful self-reinforcing flywheel. · no date given · not yet resolved

“beginning just the tip of iceberg for like what is there to come yeah yeah”

Expects: Jason Ma believes current robotics progress is only the beginning of what is yet to come, implying that further robotics progress is expected. · no date given · not yet resolved

“have a large people do distillation right like the best way to make a small model very smart is by doing distillation right you train the biggest model then you distill right so i think in robotics we're getting there but i think that's what people are going to cover shot yeah”

Expects: People will adopt the practice of training large robotics models and distilling them into smaller models. · no date given · not yet resolved

“you know you might need more than like a you know the parallel jaw gripper right you might need something more dexterous like a hand like a human hand right but then if you want to train a model on the hand then your bottleneck by the hardware stability like hand today the best hands dexterous hands are still not durable enough they overheat pretty quickly right so that becomes a bottleneck right like you we you know you quite literally have to put a”

Expects: Some robotic tasks may require a dexterous, human-like hand rather than a parallel-jaw gripper. · no date given · not yet resolved

“where the robots are joints but humans have a lot more sensors you know we hear things we feel things smell yeah you smell things right so we subconsciously use all of those to help us be so good at manipulating the physical world but today like if we're not capturing that at all then even when we scale to millions of hours you might just be missing critical dimensions of the tasks so you'll never be able to solve a task right”

Expects: Without capturing human sensory dimensions beyond current data, scaling to millions of hours might still omit critical task information and prevent robots from solving some tasks. · no date given · not yet resolved

“to figure out the other domains right i mean language is text uh in video games just video game data but for robotics like what sensors you choose determine what kind of data you have right but if you have the right set of sensors and just scale that data i think you'll get predictable return model performance so in that way robotics isn't so different so that's the better lesson as the corollary of that is that”

Expects: Model performance will improve predictably if robotics uses an appropriate set of sensors and scales the resulting data. · no date given · not yet resolved

“which is like at the end of the day, scaling will work.”

Expects: Scaling will ultimately work in robotics. · no date given · not yet resolved

“not going all the way to the final form factor because the chances that the thing you build actually doesn't last it's not good enough so you just end up scratching it anyway you know like it's public news like you know optimists you know already gone through many many iterations and they had to scratch their designs right and i think by iterating with a semi-human form factor and actually putting them out to real customers or and just getting models redeployed”

Expects: Starting with a robot's final form factor risks producing a design that is inadequate or short-lived and must be discarded. · no date given · not yet resolved

“because you're missing critical aspects of the world like sound like touch right so i think what where we could be wrong is like okay i have the best world model on vision vision-based world model but tomorrow someone else comes out with a vision plus tactile world model that's just like a hundred x more data efficient just because you know vision is a very”

Expects: A vision-and-tactile world model could achieve 100× the data efficiency of a vision-only model. · no date given · not yet resolved

“but tactile, you know, other modality might be much denser in information,”

Expects: Tactile sensing or another modality may carry denser information than vision. · no date given · not yet resolved

“So maybe having that, predicting that would just allow you to use much less vision so you can get to the same performance with much less compute, with much less data,”

Expects: Predicting tactile information could reduce the vision, compute, and data needed to reach the same performance. · no date given · not yet resolved

“a video generation companies is trying to pivot to robotics the reason is like to understand sensing is actually very complicated like we've been building that since the company started so we know the bottleneck we know how to combine them but for if you're just trying to adapt the video model to robotics action then like you know if we were wrong If, you know, video model is not enough that like you don't really have the know-how to get tactile to work very well, but we have that.”

Expects: If video models prove insufficient for robotics, video-model companies may lack the tactile-sensing expertise that the speaker's company has. · no date given · not yet resolved

“like some of a lot of ideas that I temporarily abandoned when we started dynamic because you know to build a company have to be very practical like I think simulation wasn't ready in 2024 but now with you know like the new frontier models and what we're seeing internally like this might change and that's very exciting”

Expects: The speaker believed simulation was not ready in 2024, though new frontier models and internal developments might change the situation. · no date given · not yet resolved

“Yeah, I think our vision is to truly, the high-level mission is truly bring like, you know, robots and intelligence right into the physical world like actually that means like actually seeing our robots deployed like thousands of customers even more millions of robots that's where we want to get to yeah”

Expects: Dyna Robotics aims to bring robots and intelligence into the physical world, ultimately deploying robots to thousands of customers and reaching millions of robots. · no date given · not yet resolved