How in-context learning works is not known, but it probably performs some form of gradient descent internally.

“So I guess that's just my only pushback is that who knows how in context learning works,”

“but I actually think that it's probably doing a little bit of some kind of funky gradient descent internally and that I think that that's possible.”

“So I guess I was only pushing back on you're saying it's not doing in context learning,”

“who knows what it's doing, but it's probably maybe doing something similar to it,”

“but we don't know.”