During the physical-exam task, the model's tone degraded into lay language, such as “grandpa has an unhappy heart,” which the speaker says would be unacceptable in a medical note.

“things and we kind of saw like two things the first was the model started to inflate the number of findings to kind of gain precision so it's like i'm going to generate a bunch of the same thing redundantly and i'm going to get great precision right that was one thing and so we had to kind of like update the grader to kind of constrain the model um the second thing there was like this tone degradation so the model started using these layman terms like grandpa has a has an unhappy heart we would get crucified if we put that into a medical note right so what we did is in the grader”