The speaker thinks current AI development resembles the hypothetical process: people use AI systems without understanding very well how they work.

“and then start you know plugging them into like chat products and stuff and using them in your economy um and they would be smart you know they evolved they built their own civilization in there you know so they're pretty smart and they're pretty good at generalizing and so forth um so you wouldn't understand how it works but you would still nevertheless have the ai system and and indeed that's kind of what i think is happening today for us like we don't understand how these ai systems work very well uh”