In the described update, using the policy's chosen action updates values with respect to that policy; using the optimal action updates with respect to the optimal policy and yields Q-star.

“It has the very nice property of now you uh you effectively are no longer coupled to the specific policy. And if you choose the correct uh if you choose the actions the policy chose, you're now effectively updated with respect to the policy. If you choose with the uh if you choose the optimal state, effectively you're uh you're updating with respect to the optimal policy already, have the Q_ star.”