Anthropic says it currently evaluates personalization poorly because users' instructions and preferences vary widely.

“Like, like it's, it like clearly- A model can do so many different things, and it's so depe- everyone has their own Claude MD. Uh, how do you eval that? [laughs] Yeah. We don't do a great job of it.”

“[laughs] There's like the... There's... No, seriously, it's, it's really hard. It's super hard, and they're, they're competing and contrasting. Yeah, look, look at chat. It's, it's horrible. Yeah. Yeah.”

“The constitution's helpful, but, but for things like this on how it personalizes to instructions, uh, which are different, uh, and preferences which are very, very different across these things, um, is very hard for us to do.”