The current approach leaves a gray area where Claude may object on ethical grounds to actions that are sometimes critical for aligning future AI systems.

“Whereas now there's this messy middle ground of behavior where it's like Clod is ethically objecting to something that in some cases is extremely critical to ensuring that future AI systems are well aligned.”