Fact-checking with a more powerful model augmented by web search takes a long time.

“you know you'll get some things right we do this ablation in the paper like it'll it'll uplift a little bit um for reasons we can talk about in a second but it doesn't do very well because the model like basically just goes yeah that's cool everything's fine um you can use a more powerful model and now things are really starting to get slow and expensive and that that model still has its own its own knowledge gaps or you can use a more powerful model and web search”

“and now things really take a long time so the idea that we had here was we can sort of amortize this process you know you can collect a large data set using this like model plus web search or you know in general this sort of amplified model can go out and we can like collect a data set of what the amplified model would do you know that's the model plus the web search tool”