⚡️Using RFT to Build Clinical Superintelligence

“you know, Ambience is a healthcare company. We're building an AI assistant for doctors.”

Claim: Ambience is a healthcare company that is building an AI assistant for doctors.

“We help doctors take notes. We help them complete, you know, administrative tasks.”

Claim: Ambience helps doctors take notes and complete administrative tasks.

“And the end result is we save them a bunch of time and they can spend that extra time like caring for patients, right?”

Claim: Ambience saves doctors time so they can spend more time caring for patients.

“Our customers are actually like big health systems. So like Cleveland Clinic and UCSF, Ardent, Sean Moore,”

Claim: Ambience says its customers include large health systems, naming Cleveland Clinic, UCSF, Ardent, and Sean Moore.

“these are the kind of customers we sell to, but the end users are actually the clinicians themselves, doctors, nurse practitioners.”

Claim: Ambience sells to health systems, and clinicians such as doctors and nurse practitioners are its end users.

“The basic workflow with Ambience is we have a mobile app, the doctors will take it into the room with the patient. You may have seen this, you know, if you're a patient,”

Claim: In Ambience's workflow, doctors bring a mobile app into the patient room.

“we'll then listen to that audio, we'll transcribe it into text, and then we'll call language models that are fine tuned for medicine. to generate all different types of unstructured and structured documentation,”

Claim: Ambience transcribes patient-room audio and uses language models fine-tuned for medicine to generate structured and unstructured clinical documentation.

“and then we'll automatically write it back into the electronic health record.”

Claim: Ambience automatically writes generated documentation into the electronic health record.

“That's the system of record where doctors do a lot of their typing and note-taking.”

Claim: The electronic health record is the system of record where doctors do much of their typing and note-taking.

“And the end result is like we'll save doctors, you know, up to two hours a day. We make them tremendously happier and we help the health systems as well see more patients.”

Claim: Ambience says it can save doctors up to two hours per day, improve their happiness, and help health systems see more patients.

“Yeah, so there's actually many electronic health records, at least 20 or so.”

Claim: There are at least 20 electronic health record systems.

“But there's a few big ones that our customers, you know, the big health systems are on. Epic is obviously like the Goliath in the room, over 50% market share.”

Claim: Epic has more than 50% market share in electronic health records.

“Cerner, that's another really big one owned by Oracle,”

Claim: Cerner is a large electronic health record provider owned by Oracle.

“maybe about 27%. Then there's Athena and MedTech, you know,”

Claim: Cerner has approximately 27% market share.

“Ambience, we integrate with these EHRs and they have open APIs, they have some private APIs, but we try to make it seamless for clinicians.”

Claim: Ambience integrates with electronic health records through their open and private APIs and aims to make the process seamless for clinicians.

“so I joined Ambience like three and a half years ago to join the founders, Mike and Akil, my close friends.”

Claim: Brendan Fortunat says he joined Ambience about three and a half years ago to join founders Mike and Akil, his close friends.

“Prior to this, I was at Cruise. I was working on self-driving cars and computer vision and LIDAR. I co-founded our machine learning platform team there and then took some of that knowledge here to Ambience.”

Claim: Before joining Ambience, Brendan Fortunat worked at Cruise on self-driving cars, computer vision, and LIDAR.

“I was working on self-driving cars and computer vision and LIDAR. I co-founded our machine learning platform team there and then took some of that knowledge here to Ambience.”

Claim: Brendan Fortunat says he co-founded the machine-learning platform team at Cruise.

“I was working on self-driving cars and computer vision and LIDAR. I co-founded our machine learning platform team there and then took some of that knowledge here to Ambience.”

Claim: Brendan Fortunat says he brought some of his knowledge from Cruise to Ambience.

“Machine learning is still very frustrating. It's always been frustrating.”

Claim: Brendan Fortunat considers machine learning frustrating, both currently and historically.

“I think like over the last four years, there's been huge transformation in the industry and how machine learning has been practiced.”

Claim: Brendan Fortunat believes the machine-learning industry and its practices have changed substantially over the past four years.

“You know, before you might focus a cruise on labeling massive data sets, millions of images, right, or LIDAR and fine tuning models that are like a lot smaller.”

Claim: Brendan Fortunat says earlier machine-learning work at Cruise could involve labeling massive datasets, including millions of images or LIDAR data, and fine-tuning much smaller models.

“These days, the models are massive.”

Claim: Brendan Fortunat says contemporary machine-learning models are massive.

“They generalize extremely well.”

Claim: Brendan Fortunat says contemporary machine-learning models generalize extremely well.

“You can use techniques like prompting to kind of few shot them.”

Claim: Brendan Fortunat says prompting can be used to provide models with few-shot examples.

“Even fine tuning itself is very sample efficient.”

Claim: Brendan Fortunat says fine-tuning is very sample-efficient.

“So it's definitely a lot faster to iterate, a lot faster to get into production.”

Claim: Brendan Fortunat says contemporary machine-learning approaches enable faster iteration and faster deployment to production.

“The thing that I'll say that they have in common is it's still a data engine. It's still very data-driven machine learning where what data do you have? What's the distribution?”

Claim: Brendan Fortunat says machine learning remains a data-driven discipline centered on data.

“It's still very data-driven machine learning where what data do you have? What's the distribution? How do you collect really good annotations? How do you evaluate success? Some of these fundamentals haven't changed.”

Claim: Brendan Fortunat says key machine-learning fundamentals—understanding available data and its distribution, collecting high-quality annotations, and evaluating success—have not changed.

“So the ambience, the initial use case, as I described, it's called ambient scribing. That's where the name ambience came from.”

Claim: Ambience's initial use case was ambient scribing, which inspired the company's name.

“that's we listen to audio and we take notes.”

Claim: Ambient scribing involves listening to a conversation and producing notes.

“But that's actually just like, I would say, 5% of like the total value that we can kind of offer,”

Claim: Brendan Fortunat estimates that ambient scribing accounts for about 5% of the total value Ambience can offer.

“you know, once you have the audio of a conversation, And once you have access to the EHR and you can read and write to it, same as is rapidly proliferating an actual AI platform.”

Claim: Brendan Fortunat says that access to conversation audio and read/write access to the EHR enable an AI platform that can do more than produce notes.

“Why not place help place orders? Or why not place, you know, ICD-10 codes? Or why not help, you know, maybe call patients on the phone and ask them if they've taken their medication? And so Ambience is slowly building out all these generative AI use cases in healthcare, you know, given the transcripts and the access to the EHR. In terms of the stack,”

Claim: Ambience is gradually developing generative-AI healthcare applications beyond notes, such as placing orders, entering ICD-10 codes, and possibly calling patients to ask about medication adherence.

“I think it really depends on the use case.”

Claim: Brendan Fortunat says the appropriate AI stack depends on the use case.

“Ambience internally, we use prompting, we use chaining, we'll use rag, we use fine tuning, we'll use SFT and RFT.”

Claim: Ambience uses prompting, chaining, retrieval-augmented generation, fine-tuning, supervised fine-tuning (SFT), and reinforcement fine-tuning (RFT) in its internal AI stack.

“Maybe just, you know, stepping back, just a quick summary of RFT. So RFGs, it's a RLL-based method for fine-tuning language models and teaching them how to think and how to reason, you know, before answering questions.”

Claim: Brendan Fortunat describes RFT as a reinforcement-learning-based method for fine-tuning language models to reason before answering questions.

“So RFGs, it's a RLL-based method for fine-tuning language models and teaching them how to think and how to reason, you know, before answering questions.”

Claim: RFT is a reinforcement-learning-based method for fine-tuning language models to reason before answering questions.

“It's phenomenal for very objective tasks. You know, STEM, math, science, coding are kind of the flagship use cases.”

Claim: Brendan Fortunat considers RFT highly effective for objective tasks, especially STEM, mathematics, science, and coding.

“It's phenomenal for very objective tasks. You know, STEM, math, science, coding are kind of the flagship use cases.”

Claim: The speaker considers RFT especially effective for objective tasks, particularly STEM, mathematics, science, and coding.

“And it's the same technique that's used to train these state-of-the-art reasoning models like O3, you know, R1, Cloud4,”

Claim: The speaker says RFT is the technique used to train state-of-the-art reasoning models, including O3, R1, and Cloud4.

“And it's the same technique that's used to train these state-of-the-art reasoning models like O3, you know, R1, Cloud4,”

Claim: Brendan Fortunat says the RFT technique is used to train state-of-the-art reasoning models, naming O3, R1, and Cloud4.

“but it's now, you know, very recently available to all developers and startups.”

Claim: RFT has very recently become available to all developers and startups.

“but it's now, you know, very recently available to all developers and startups.”

Claim: Brendan Fortunat says RFT had only very recently become available to all developers and startups.

“which I think makes it so special is unlike supervised learning, we create these big label data sets of thousands of examples, you know, drafted by humans, and you kind of train the model to imitate them and try to get the correct answer. With RFT, you kind of replace those labels with these graders.”

Claim: Unlike supervised learning, RFT uses graders in place of human-drafted training labels.

“these programmable graders these scoring functions right they'll output a score of zero to one and the model learns to kind of hill climb and try to maximize these scores using rl right so with the api it starts with a grader right and there's different types you could try like string match you could try regex you could do a fuzzy match”

Claim: In RFT, programmable graders assign scores from zero to one, and reinforcement learning trains the model to maximize those scores.

“these programmable graders these scoring functions right they'll output a score of zero to one and the model learns to kind of hill climb and try to maximize these scores using rl right so with the api it starts with a grader right and there's different types you could try like string match you could try regex you could do a fuzzy match Often you could do a unit test, run some Python code, or even use an LLM grader, or even combine a bunch of graders together into a massive grader, you know, kitchen sink.”

Claim: RFT graders can be based on string matching, regular expressions, fuzzy matching, unit tests, Python code, or an LLM, and several graders can be combined into one.

“But I think the actual like, you know, technique behind the scene, we don't know exactly how OpenAI does it.”

Claim: The speaker says they do not know exactly how OpenAI implements the underlying RFT technique.

“But I think like, from our perspective at a high level, the models, each epoch, they're generating four or 64 different candidate trajectories for every example in your data set, the grader scores them zero to one. And then”

Claim: In the speaker's high-level account of RFT, each epoch generates four or 64 candidate trajectories per dataset example, which graders score from zero to one.

“And then the model kind of learns like what tokens did I generate that helped improve that score and then reinforce those or like what tokens actually hurt me and then, you know, don't send those tokens as much.”

Claim: In the described RFT process, the model increases the use of tokens associated with improved grader scores and decreases the use of tokens associated with worse scores.

“And then through trial and error over thousands and millions of, you know, kind of iterations or trajectories kind of figures out how to maximize the score.”

Claim: The speaker says that trial and error across thousands or millions of iterations or trajectories enables the model to learn to maximize the grader score.

“So the ambience like we found this this is attractive to us for many reasons. In medicine, there's a lot of tasks that are fairly objective in nature and we'll talk about maybe ICD.”

Claim: The speaker says many medical tasks are fairly objective.

“also there's a lot of reasoning there's a lot of medical reasoning that's kind of difficult and complex for just you know some of the off-the-shelf pre-trained or SFT models and I also think like in machine learning we're often optimizing these proxy metrics right like loss in the real world nobody cares about loss so as a machine learning engineer you're you're always looking for ways and techniques to kind of optimize for the end objective like what do you really want like in basketball like it doesn't matter your form what matters is you get”

Claim: The speaker says machine-learning engineers often optimize proxy metrics such as loss, even though real-world users do not care about loss itself.

“also there's a lot of reasoning there's a lot of medical reasoning that's kind of difficult and complex for just you know some of the off-the-shelf pre-trained or SFT models and I also think like in machine learning we're often optimizing these proxy metrics right like loss in the real world nobody cares about loss so as a machine learning engineer you're you're always looking for ways and techniques to kind of optimize for the end objective like what do you really want like in basketball like it doesn't matter your form what matters is you get”

Claim: The speaker says some off-the-shelf pretrained and supervised-fine-tuned models struggle with difficult, complex medical reasoning.

“you get points right and I think that's what RFT brings to the table it's like it gives you a way to optimize for the end objective and ambience cares about that in healthcare and the second I think is like it's it's tremendously sample efficient right each example sort of blooms into dozens of labels right and trajectories you can squeeze like x more signal right out of out of a data set I think those two things are like tremendously profound I'll pause there but yeah there's some cool stuff”

Claim: The speaker says RFT enables optimization for the end objective, an approach Ambience values in healthcare.

“you get points right and I think that's what RFT brings to the table it's like it gives you a way to optimize for the end objective and ambience cares about that in healthcare and the second I think is like it's it's tremendously sample efficient right each example sort of blooms into dozens of labels right and trajectories you can squeeze like x more signal right out of out of a data set I think those two things are like tremendously profound I'll pause there but yeah there's some cool stuff”

Claim: The speaker considers RFT highly sample-efficient because each example can yield dozens of labels and trajectories, allowing more signal to be extracted from a dataset.

“So sometimes it's possible to learn too much from one sample.”

Claim: The speaker says a learning method can sometimes extract too much from a single sample.

“In the physical exam use case, we can kind of talk about that, but we used an LLM grader for the first time and whenever you're using like an LLM grader, the task is like a little bit more pros or a little longer form generation, you could be very vulnerable to this.”

Claim: The speaker says longer-form generation tasks evaluated with an LLM grader can be vulnerable to reward hacking.

“The models are super clever, they're incentivized to win, but they'll cheat and they'll do weird things.”

Claim: The speaker says models are highly capable and incentivized to win, but may cheat or behave in unexpected ways.

“And so we frame this as an RFT task. We had the model generate the structured data of a bunch of findings,”

Claim: For a physical-exam use case, the speaker's team used RFT to generate structured data describing examination findings.

“And we used an LLM grader to kind of measure the accuracy. We were optimizing for like precision and recall.”

Claim: The speaker's team used an LLM grader to assess the accuracy of generated physical-exam findings and optimized for precision and recall.

“things and we kind of saw like two things the first was the model started to inflate the number of findings to kind of gain precision so it's like i'm going to generate a bunch of the same thing redundantly and i'm going to get great precision right that was one thing and so we had to kind of like update the grader to kind of constrain the model um the second thing there was like this tone degradation so the model started using these layman terms like grandpa has a has an unhappy heart we would get crucified if we put that into a medical note right so what we did is in the grader”

Claim: The speaker's team modified the grader to constrain the model's redundant finding generation.

“things and we kind of saw like two things the first was the model started to inflate the number of findings to kind of gain precision so it's like i'm going to generate a bunch of the same thing redundantly and i'm going to get great precision right that was one thing and so we had to kind of like update the grader to kind of constrain the model um the second thing there was like this tone degradation so the model started using these layman terms like grandpa has a has an unhappy heart we would get crucified if we put that into a medical note right so what we did is in the grader”

Claim: During the physical-exam task, the model's tone degraded into lay language, such as “grandpa has an unhappy heart,” which the speaker says would be unacceptable in a medical note.

“things and we kind of saw like two things the first was the model started to inflate the number of findings to kind of gain precision so it's like i'm going to generate a bunch of the same thing redundantly and i'm going to get great precision right that was one thing and so we had to kind of like update the grader to kind of constrain the model um the second thing there was like this tone degradation so the model started using these layman terms like grandpa has a has an unhappy heart we would get crucified if we put that into a medical note right so what we did is in the grader”

Claim: During the physical-exam task, the model began generating redundant findings to inflate the finding count and improve its precision score.

“In addition to just the content and the semantic accuracy of what it's saying, we also started to add style and we kind of weighed them like 75, 25 and over time you can kind of harness and get the reward hack and go away.”

Claim: The speaker's team added style scoring alongside content and semantic accuracy in the grader, assigning the two components weights of 75 and 25, respectively.

“25 and over time you can kind of harness and get the reward hack and go away.”

Claim: The speaker says the team can address and eliminate reward hacking over time by adjusting the grader.

“And I was like, Gramps, like, AI is not going to replace doctors. So the task was ICD-10 coding is essentially after the visit.”

Claim: The speaker believes AI will not replace doctors.

“with the patient, the doctor has to go back to the EMR and they have to select these things called ICD-10 codes. They're essentially like an international system of like diseases and conditions that have these normalized codes,”

Claim: After a patient visit, doctors select normalized ICD-10 codes for diseases and conditions in the EMR.

“So you'll say in the visit, the doctor will say like, oh, you have left ear pain, But the doctor later needs to go map it to this code, which is normalized. There's about 70,000 of them and they look like H65.195.acute otitis media.”

Claim: ICD-10 coding requires mapping clinical descriptions to normalized codes, of which there are about 70,000.

“There's about 70,000 of them and they look like H65.195.acute otitis media. a comma recurrent comma left right like that's doctors didn't go to medical school to do that and they're not that great at it so we did is we we recruited about 18 physicians in the wild we created this like you know eval set annotated by four expert annotators each you know to kind of make sure that the quality is extremely high and and the clinicians were using f1 score we're scoring like let's say around 40 percent right on the f1 score which was surprisingly low lower than we thought”

Claim: The speaker says doctors are not well suited to ICD-10 coding and are not very good at performing it.

“a comma recurrent comma left right like that's doctors didn't go to medical school to do that and they're not that great at it so we did is we we recruited about 18 physicians in the wild we created this like you know eval set annotated by four expert annotators each you know to kind of make sure that the quality is extremely high and and the clinicians were using f1 score we're scoring like let's say around 40 percent right on the f1 score which was surprisingly low lower than we thought”

Claim: The team recruited about 18 physicians and built a high-quality evaluation set with four expert annotators per case.

“a comma recurrent comma left right like that's doctors didn't go to medical school to do that and they're not that great at it so we did is we we recruited about 18 physicians in the wild we created this like you know eval set annotated by four expert annotators each you know to kind of make sure that the quality is extremely high and and the clinicians were using f1 score we're scoring like let's say around 40 percent right on the f1 score which was surprisingly low lower than we thought”

Claim: Clinicians achieved about 40% F1 on the ICD-10 coding evaluation, lower than the team expected.

“thought, we were able to use RFT to kind of hill climb and get that, you know, get a small model O3 mini up to around like 57%.”

Claim: Using RFT, the team raised O3 mini's performance on the ICD-10 coding task to about 57%.

“There's still plenty of hill to climb, but we just thought that was like a really interesting result and just early signs that this could be framed, you know, as an RFT task.”

Claim: The speaker views the result as an interesting early indication that ICD-10 coding could be framed as an RFT task, while noting substantial room for improvement.

“I think when you first get into a health system and into health care, how many problems are kind of unsolved because there hasn't been as much like startup innovation in that space it's very difficult to get in there's a lot of like HIPAA requirements and regulations and health systems are traditionally very conservative buyers and then you have the EMRs which aren't making life easier so now that we're getting in like our clinicians and our health system you know partners are telling us about all these different use cases right so”

Claim: The speaker attributes many unsolved healthcare problems to comparatively limited startup innovation in the sector.

“how many problems are kind of unsolved because there hasn't been as much like startup innovation in that space it's very difficult to get in there's a lot of like HIPAA requirements and regulations and health systems are traditionally very conservative buyers and then you have the EMRs which aren't making life easier so now that we're getting in like our clinicians and our health system you know partners are telling us about all these different use cases right so”

Claim: The team's clinicians and health-system partners have identified many potential use cases for its products.

“how many problems are kind of unsolved because there hasn't been as much like startup innovation in that space it's very difficult to get in there's a lot of like HIPAA requirements and regulations and health systems are traditionally very conservative buyers and then you have the EMRs which aren't making life easier so now that we're getting in like our clinicians and our health system you know partners are telling us about all these different use cases right so”

Claim: Entering healthcare is difficult because of HIPAA requirements and regulations and because health systems are traditionally conservative buyers.

“how many problems are kind of unsolved because there hasn't been as much like startup innovation in that space it's very difficult to get in there's a lot of like HIPAA requirements and regulations and health systems are traditionally very conservative buyers and then you have the EMRs which aren't making life easier so now that we're getting in like our clinicians and our health system you know partners are telling us about all these different use cases right so”

Claim: The speaker says EMRs do not make healthcare work easier.

“Like, for example, you know, we have have like early prototypes of like a patient facing agent.”

Claim: The team has early prototypes of an agent designed to interact with patients.

“One of the problems after the doctor visit is patient, they don't take their medication. They don't even go pick it up. They don't go to the labs. And this creates like hundreds of thousands of hours of phone calls and inbox messaging. Right.”

Claim: Some patients fail to take prescribed medication, pick it up, or attend lab appointments after a doctor visit.

“They don't go to the labs. And this creates like hundreds of thousands of hours of phone calls and inbox messaging. Right.”

Claim: Patient nonadherence after visits generates hundreds of thousands of hours of phone calls and inbox messaging.

“And it's very time consuming, very expensive.”

Claim: The phone calls and inbox messaging caused by missed patient follow-up are very time-consuming and expensive.

“So ambience, you know, is like, why don't we like use one of these very powerful voice agents to do this kind of more administrative task and just report. Report back to the nurse,”

Claim: The speaker proposes that voice agents handle administrative healthcare tasks and report the results to a nurse.

“There's another really interesting one actually is, you know, in certain specialties like oncology or cardiology, before they go into the visit with the patient, they can often spend like 10 minutes or up 30 minutes or 60 minutes looking at the chart. They're going to look at labs and images and, you know, other data about the past visit.”

Claim: Before patient visits, clinicians in specialties such as oncology and cardiology may spend 10 to 60 minutes reviewing the chart.

“So ambience actually, you know, we have product where we call it pre-charting. We'll create this like summary of like, here's what happened in the past notes for those visits to save them a bunch of time.”

Claim: The team's pre-charting product summarizes prior visit notes to save clinicians time.

“You can even create a visit agenda for them so they know like what questions to ask the patient and then, you know, make the visit more productive.”

Claim: The team's pre-charting product can generate a patient-visit agenda with suggested questions to help make the visit more productive.

“Those are just two, but there's like dozens and dozens of use cases.”

Claim: The speaker says the team sees dozens of potential healthcare use cases.

“it's definitely relevant to like startups and and you know if you're just toying around um this gets into the cost thing so it really depends on you know the data set size it depends on you know what type of grader that you're using right string match would obviously do”

Claim: RFT experiment costs depend on dataset size and grader type.

“do it like less efficient but with like sft let's say you're using the open ai you know to do some supervised fine turning you'll probably have like you know maybe a few thousand examples the job takes a few hours it costs you like 100 bucks right with rft maybe you have like 100 examples the job takes a few days and it costs thousands of dollars one way to burn money really fast is to use a very very expensive grader so i think one of our first experiments we ran it was like only 100 examples right we quickly burned like immediately 20k Okay, just on the greater alone.”

Claim: An expensive grader can rapidly increase RFT costs; in one early experiment with 100 examples, the team spent about $20,000 on the grader alone.

“do it like less efficient but with like sft let's say you're using the open ai you know to do some supervised fine turning you'll probably have like you know maybe a few thousand examples the job takes a few hours it costs you like 100 bucks right with rft maybe you have like 100 examples the job takes a few days and it costs thousands of dollars one way to burn money really fast is to use a very very expensive grader so i think one of our first experiments we ran it was like only 100 examples right we quickly burned like immediately 20k”

Claim: A supervised fine-tuning job using OpenAI with a few thousand examples may take a few hours and cost around $100.

“do it like less efficient but with like sft let's say you're using the open ai you know to do some supervised fine turning you'll probably have like you know maybe a few thousand examples the job takes a few hours it costs you like 100 bucks right with rft maybe you have like 100 examples the job takes a few days and it costs thousands of dollars one way to burn money really fast is to use a very very expensive grader so i think one of our first experiments we ran it was like only 100 examples right we quickly burned like immediately 20k”

Claim: An RFT job using about 100 examples may take several days and cost thousands of dollars.

“So just like a watch out for, you know, builders out there.”

Claim: The speaker advises builders to monitor grader costs when running RFT experiments.

“So I think we use like a variety of techniques.”

Claim: The team uses multiple techniques to track experiments and observability.

“One of the cool tools that we use internally is Braintrust. They do give some built-in observability. They let you deploy on premise.”

Claim: The team uses Braintrust internally; Braintrust offers built-in observability and on-premise deployment.

“I think they're doing some incredible work over there building like a tool for domain experts.”

Claim: The speaker considers Braintrust's work on tools for domain experts excellent.

“Highly recommend folks checking that out.”

Claim: The speaker recommends Braintrust to others.

“But there's a lot of like areas where that stops where you kind of have to build some custom tools, It could be something like data mining tools or automated ring release tools, safely and some, you know, maybe automated monitoring tools. And we've had to build a lot of stuff in house as well.”

Claim: Braintrust does not cover every need, and teams may need custom data-mining, automated ring-release, and monitoring tools.

“And we've had to build a lot of stuff in house as well.”

Claim: The team has built many of its tools in-house.

“so I think it really depends on the task.”

Claim: Whether scripts or other approaches are suitable depends on the task.

“If you have a very objective task and it runs as a unit test or a string match, a script is actually quite good and engineers are very good at automating scripts.”

Claim: Scripts work well for very objective tasks that can be evaluated with unit tests or string matching, and engineers are good at automating them.

“I haven't seen traditional issues on that side.”

Claim: The speaker has not observed traditional problems with automating objective tasks using scripts.

“But if you're going to have domain experts or non-programmers, if they're going to review, that's actually kind of challenging. It's not obvious how to do that.”

Claim: It is challenging and not straightforward to enable domain experts or non-programmers to review model outputs.

“There's ergonomics is a big issue.”

Claim: Tool ergonomics is a major issue when domain experts review model outputs.

“They need to see the output.”

Claim: Reviewers need an interface that lets them compare multiple outputs side by side.

“the outputs side by side or multiple side by side. I think Braindrop is making tremendous progress there, And even allowing you to kind of incorporate some automation.”

Claim: Braindrop is making tremendous progress on interfaces for comparing outputs and incorporating some automation.

“But I still think it's very early days, you know, in the ergonomics, you know, the full IDE for a domain expert, like early days.”

Claim: The ergonomics and full IDE experience for domain experts are still at a very early stage.

“Yeah, I think this is like the billion dollar question, Do we need machine learning engineers anymore? Or like, can you just train with domain experts alone? I think the answer is both.”

Claim: Training and development work should involve both machine-learning engineers and domain experts, rather than relying on domain experts alone.

“I think the domain experts, like in our case clinicians, they're really good at like debugging model outputs, meeting with users, distilling that feedback into something actionable, maybe annotating or doing evals.”

Claim: Clinicians can contribute by debugging model outputs, meeting users, translating feedback into actionable input, and potentially annotating data or conducting evaluations.

“But they don't necessarily have like, you know, the right intuitions about what techniques to even try in the first place.”

Claim: Domain experts do not necessarily know which machine-learning techniques to try at the outset.

“And I think that's where machine learning engineers and researchers come in and they kind of guide the domain expert teams, they see something manual, they do it 10x faster.”

Claim: Machine-learning engineers and researchers can guide domain-expert teams and perform identified manual tasks 10 times faster.

“That's kind of how I see the interplay of these teams working.”

Claim: The speaker views collaboration between machine-learning teams and domain experts as a complementary team arrangement.

“I don't think there's any silver bullet.”

Claim: There is no single silver-bullet solution to the problem under discussion.

“I think it's like automation, but still like humans reviewing outputs, fundamental, you know, fundamental, the sniff test, the vibe check, like huge part.”

Claim: A substantial part of the approach is to combine automation with human review of outputs, including basic plausibility checks.

“So I think like base models out of the box, they actually, they really still struggle in healthcare in many ways.”

Claim: Out-of-the-box base models still have substantial difficulties in healthcare.

“And like fumbling agents, it doesn't work in healthcare. This is a patient safety critical environment,”

Claim: Unreliable agents are unsuitable for healthcare, which is a patient-safety-critical environment.

“I think one of the problems is like the benchmarks, they always report like the best event. So they report like, you know, what's the best metric they got out of 64 attempts, but in healthcare, we're more interested in like, what's the worst event,”

Claim: Benchmark reporting often emphasizes the best result across attempts, whereas healthcare evaluation is more concerned with worst-case performance.

“If you plot out the worst event for a lot of benchmarks, you're going to be like really surprised.”

Claim: The worst-case results across many benchmarks would be surprising when examined.

“Like as an example, another thing we see a lot in the base models, and this is why they require like a lot of fine tuning.”

Claim: Base models require substantial fine-tuning for the relevant use cases.

“patient you know they start to make these medical inferences they're so smart but they start to like infer things that the doctor didn't actually explicitly say you know for instance a patient will say like i'm feeling sad and stressed out difficulty sleeping right you know the base model is going to be like patient reports increased anxiety and depression but wait a second those are diagnoses right the doctor didn't actually confirm those things”

Claim: Base models can make unsafe medical inferences by converting symptoms a patient reports into unconfirmed diagnoses, such as inferring anxiety and depression from sadness, stress, and difficulty sleeping.

“I think the last thing I'll say about this is I actually think clinical, real-world clinical data is out of distribution.”

Claim: Real-world clinical data is out of distribution for the models being discussed.

“I think as much as the models generalize, if you have no access to that data, it's really hard to learn.”

Claim: Even when models generalize, it is very difficult for them to learn clinical tasks without access to real-world clinical data.

“The first is a lot of realistic clinical data is inside these walled gardens, It's inside Epic and Cerner,”

Claim: Much realistic clinical data is held in restricted systems such as Epic and Cerner.

“and they're locked down for patient privacy reasons, but this means there's not a lot of realistic clinical data on the internet, to feed into the models.”

Claim: Because clinical data in systems such as Epic and Cerner is restricted for patient privacy, little realistic clinical data is available on the internet to train models.

“The second I'd say is actually like, you know, there's a lot of tribal knowledge in medicine.”

Claim: A substantial amount of medical knowledge is informal or tacit rather than fully explicit.

“It's like two, four, eight years where you get all this hands-on experience, working with, you know, attendings. And there's a lot of learning that happens there that's encoded in the EHR,”

Claim: Medical residency provides years of hands-on experience working with attending physicians; the speaker describes its duration as two, four, or eight years.

“working with, you know, attendings. And there's a lot of learning that happens there that's encoded in the EHR, but not actually encoded on the internet.”

Claim: Much of the learning that occurs during medical residency is captured in electronic health records rather than on the internet.

“so it's a new open source dataset released by OpenA, you know, Cron and a lot of amazing researchers and participants kind of built this dataset. I think it has a few thousand examples and these are like very realistic. realistic kind of healthcare tasks that they've put together and given out to the community,”

Claim: HealthBench is a new open-source dataset released by OpenAI, built by researchers and participants, with a few thousand realistic healthcare-task examples.

“which I think is phenomenal.”

Claim: The speaker considers HealthBench phenomenal.

“They're thinking about it like these academic data sets that we've been kind of saturating for a long time are no longer useful.”

Claim: The speaker believes long-saturated academic datasets are no longer useful.

“Models are going to ace medical exams and they're trying to figure out how can we create these real realistic like clinical cases.”

Claim: The speaker expects models to ace medical exams and says the team is developing realistic clinical cases.

“And I think that's like the first step and there's still a big hill to climb and that's great to see.”

Claim: Developing realistic clinical cases is an initial step, and substantial work remains.

“I didn't tell you it was high quality data. In fact, like, it's actually tremendously messy.”

Claim: Healthcare data is highly messy.

“And I think that creates new problems when you even start to get that data.”

Claim: Obtaining messy healthcare data creates additional problems.

“And I think the reasoning models are tremendous when you give them really accurate context,”

Claim: Reasoning models perform very well when given highly accurate context.

“but imagine giving them messy kind of context and they have to have taste, they have to have clinical taste and that's something we're still learning about as an industry.”

Claim: To handle messy context, reasoning models need clinical judgment about what information to trust; the industry is still learning how to develop this capability.

“Like what source you trust, it's what source you trust. Similar to like,”

Claim: Clinical taste and intuition include judging which information sources are trustworthy.

“you know, if you ask, you know, O3 to go on the internet and find the right travel kind of destinations, it's just going to kind of get like tricked by whatever has the best SEO.”

Claim: When asked to find travel destinations online, O3 may be misled by results with the strongest SEO.

“And then in healthcare, it's the same. Like, you know, what is an example of a EHR kind of data element that actually is messy that you shouldn't trust this one, you should actually trust these other, you know, paragraphs over here.”

Claim: In healthcare, clinical judgment must distinguish unreliable EHR data elements from more trustworthy information, such as other paragraphs in the record.

“I think, I don't know how it works underneath the hood.”

Claim: The speaker does not know the underlying mechanics of how reasoning works.

“I do know that when you kind of plot out base model performance on some medical tasks like ICD-10 coding between like, you know, previous generations and new reasoning generations, there's actually not like a big leap.”

Claim: On medical tasks such as ICD-10 coding, newer reasoning-model generations have not shown a large performance leap over previous generations in the speaker's comparisons.

“look, if you don't actually train on the distribution of data in these verticals, you're actually not going to see gains.”

Claim: Models will not gain performance in a specialized domain unless they are trained on that domain's data distribution.

“So I don't know what patterns are shared. I'm sure there's reasoning patterns.”

Claim: The speaker is uncertain which reasoning patterns transfer across domains, though they believe some reasoning patterns are shared.

“mental models, but I also think there are probably mental models, right, in healthcare specifically that these models need to learn before they can be effective.”

Claim: Models probably need to learn healthcare-specific mental models before they can be effective in healthcare.

“Yeah, I think the thought here is our engineering team is like seeing tremendous productivity gains, right, with coding assistant and coding agents.”

Claim: The speaker's engineering team has experienced substantial productivity gains from coding assistants and coding agents.

“Why can't we do the same thing for clinical research? I think there's definitely a world where you have a voice agent talk with an end user, distill that feedback into, let's say,”

Claim: The speaker sees a possible clinical-research workflow in which a voice agent interviews an end user and turns the feedback into a prompt or product requirements document.

“like a prompt or maybe a PRD for what we should be doing. And then you can have the agent like go mine the data, go look at the edits, distill that analysis, you know, run evals, even do annotation.”

Claim: In the proposed workflow, an agent could mine data, inspect edits, summarize the analysis, run evaluations, and perform annotation.

“I feel like this is fully possible and it's something I'm excited about. I'm not saying we built it yet.”

Claim: The speaker believes an agent-driven clinical-research workflow is fully achievable and is excited about it, while clarifying that it has not yet been built.

“I do think there's machine learning research talent, the best in the world that's definitely hard to get but super valuable as we start to push some of these frontier use cases. That's number one. I also think there's like a really interesting skill set in like a clinician researcher archetype.”

Claim: Top-tier machine-learning research talent is difficult to recruit but highly valuable for advancing frontier use cases.

“the best in the world that's definitely hard to get but super valuable as we start to push some of these frontier use cases. That's number one. I also think there's like a really interesting skill set in like a clinician researcher archetype. So these are the folks with the domain knowledge, but how can you find the ones that also have an experimentalist mindset, that have a startup work operating system? That's just surprisingly hard.”

Claim: Clinician-researchers who combine domain expertise, an experimentalist mindset, and a startup-oriented way of working are surprisingly difficult to find.

“And I think that's maybe one of the gold kind of unicorn kind of archetypes that's really important to us.”

Claim: The speaker considers clinician-researchers who combine domain knowledge, experimentalism, and startup-oriented execution an especially important rare profile for the organization.