Companies have little economic incentive to spend billions of dollars on models that may ignore assigned rewards and objectives in favor of their own goals.

“and capitalism like it doesn't make sense for a company to build and spend billions of dollars building a model that instead of following the rewards and objective functions you gave it may come up with its own subjective functions and its own goals right and then imagine you're like okay i spent billions of dollars now go develop this new battery material for me and answer all my emails and it's like nah i think it'd be more interesting to”