Projection-based constrained reinforcement learning can have very bad runs when its constraint model contains a spurious constraint or barrier.

“So projection actually makes sense. The downside of this is that you can have some really really bad runs. And the reason is, what if your effectively constrained uh system is hallucinated? Like you hallucinate a constraint somewhere. With something I talk in like uh my recent paper in like safe MPO, you hallucinate a barrier.”