Joscha Bach says an LLM can generate arbitrary characters and stories, making it difficult to determine whether its output is merely imitation or reflects simulated causal experience.

“And that, that is one of the big difficulties, right? The LLM is a shape-shifter. It can produce arbitrary characters and stories. And so at which point is it fake, and at which point is it actually simulating the causal experience?”