A continuing limitation of frontier LLMs is reliably reasoning over a massive text input and locating the relevant information throughout it, rather than focusing only on its beginning and end.

“the limiting factor has been and will continue to be, although it's getting a lot better over time,”

“the ability for these frontier LLMs to be able to take a massive wall of text and not just read the first three lines and the last three lines,”

“but actually be able to properly reason and like localize themselves into,”

“We are halfway through this procedure and these things have already happened and these things haven't happened yet and these things are true and these things are false about the situation and so therefore like this is the one thing that I should be really zooming in on right now, right?”