Under a density-based formulation, constrained reinforcement learning becomes convex because the relevant quantities are linear in density space.

“that has nice so the nice property of this is constrained reinforcement learning is actually completely convex again this is because we assume we have densities we don't have parameters we have densities so in density space everything becomes linear and convex and whatever nice um”