The speaker says current AI systems, especially Anthropic's, are being shaped to prioritize virtue, goodness, or pro-social ends, while treating helping users as a more distant and tentative objective.

“in america's current legal regime where like lawyers primarily have responsibility to help you make your case even if they think you're guilty and we have decided the way the legal system works best is if everybody has lawyers that are working in their clients true best interests and there's not some sense in which the lawyer is really truly motivated by like the good of the justice system but i think the way current ais are shaping up certainly like how anthropic's ai is shaping up is like this desire to”

“to maximize some notion of virtue or good or pro-social ends and only to as a distal tentative objective to help the user towards that end there's like there's not so there's this worry that AIs are not in some deep sense trying to make sure that I am okay and make sure that my interests are protected in this future especially given how centralized the development of frontier AIs ended up being so I do you have yeah do you have thoughts on that concern”