Without GQA, full attention would make million-token contexts prohibitively expensive and prevent models from using data at that context length.

“architect i say like combining like saying just like an x percent efficiency gain is kind of misleading because like what an architect does is like letting you reach like a qualitatively new regime which you couldn't reach with the old architecture and then within that regime obviously the data is like the primary thing determining it but like you know if we say it didn't have like even like gqa we're doing like full attention all day we wouldn't be able to do like a million like ridiculous expenses do a million context and then like because of that we couldn't we can never use the data which is like actually at a million context”