In autonomous settings, an LLM needs explanatory framing for a heuristic such as UCB to use it coherently; simply instructing it to use UCB is insufficient.

“So that's part of the reason why some heuristics are required,”

“because it's not like you can just say, oh, use UCB.”

“There has to be some kind of framing to explain why you should use it. And then the LLM becomes coherent in the autonomous setting.”