Under a density-based formulation, constrained reinforcement learning is convex because the problem is linear in density space rather than parameter space.

“that has nice so the nice property of this is constrained reinforcement learning is actually completely convex again this is because we assume we have densities we don't have parameters we have densities so in density space everything becomes linear and convex and whatever nice um”