Handshake investigates whether a model's failure to improve on a combined visual-and-reasoning task is due to image understanding or reasoning about the image description.

“And there are two interesting things from that. One is that how is it climbing or not climbing? So for instance, if we see that the model has a visual task and a reasoning task associated with it and, and the model doesn't climb, is it because it failed to understand the image or is it that it failed to understand the, um, reasoning on how to reason the, the description of the image?”