During the physical-exam task, the model began generating redundant findings to inflate the finding count and improve its precision score.

“things and we kind of saw like two things the first was the model started to inflate the number of findings to kind of gain precision so it's like i'm going to generate a bunch of the same thing redundantly and i'm going to get great precision right that was one thing and so we had to kind of like update the grader to kind of constrain the model um the second thing there was like this tone degradation so the model started using these layman terms like grandpa has a has an unhappy heart we would get crucified if we put that into a medical note right so what we did is in the grader”