Anyone who has finished an emergency shift knows the chart stack. The department is still loud, the next team has the board, and a column of unsigned notes waits on the workstation. You read each one, fix what’s wrong, and sign. That signature has always meant something specific. I saw this patient, and this record is mine.
Ambient AI scribes change who writes the first draft. A microphone picks up the visit, software turns the conversation into a structured note, and the clinician reviews it before it goes into the chart. Health systems are moving quickly on this, and the reason is easy to respect. Documentation eats hours that should go to patients, and a lot of it spills into evenings at home.
The question I keep putting to leaders is narrower. When a clinician signs a note a machine drafted, what did they actually check?
What the early evidence shows
The Permanente Medical Group turned on ambient AI scribing for 10,000 physicians and staff in October 2023. In the first 10 weeks, 3,442 physicians used it in as many as 303,266 encounters.[1] Reviewers scored a sample of 35 AI-generated notes with a modified note-quality instrument, and the average came out at 48 of a possible 50.[1]
That’s a strong score. The same report also describes the misses. In one case a physician mentioned scheduling a prostate exam, and the summary said the exam had been performed. In another, the physician talked about issues with the patient’s hands, feet and mouth, and the summary recorded a diagnosis of hand, foot and mouth disease. Some summaries left details out, such as chest pain and anxiety assessments.[1] The authors also wrote that they couldn’t directly verify what edits physicians made or what proportion of the AI text got revised.[1]
Then came a randomized trial. At UCLA, 238 outpatient physicians in 14 specialties were assigned to one of two AI scribe products or to usual care for about two months. One product cut time spent writing each note by 9.5 percent compared with control. The other showed no significant change. Both scribe groups reported possible improvements in burnout and task load, which the authors said need confirmation in larger trials. Physicians rated clinically significant inaccuracies as happening “occasionally,” and the authors concluded that ongoing vigilance is required.[2]
Put those side by side and the picture is fair. The notes are usually good. Sometimes they’re wrong in ways that matter. Review is the step that’s supposed to catch those errors, and few programs measure it.
Why a good note is harder to check
A sloppy draft invites editing. A clean one invites a signature.
Kahneman described the halo effect, where a general favorable impression shapes our judgment of qualities we never actually looked at.[3] An AI note shows up organized, grammatical and already sitting in the right template. None of that polish tells you whether the exam it describes took place. A reviewer reading for overall quality can approve a note with one invented finding in it, because everything around that finding reads right.
Look at the type of errors in the Permanente report. A planned exam became a completed exam. A symptom became a diagnosis. Both can change what the next clinician believes happened, and both are hard to spot when you’re reading for flow.
Omissions are worse. Nobody notices a missing sentence unless they remember the visit well enough to miss it. At the end of a long shift, looking at notes from patients seen hours earlier, that memory is thin.
Responsibility didn’t move
Razmi makes the point plainly in AI Doctor: relying on an AI output doesn’t remove the clinician’s responsibility for the clinical decision.[4] The Permanente training materials said the same about notes. They told physicians they stay responsible for reviewing and revising all documentation, whether or not a scribe helped.[1]
I agree with both. I’d add that a responsibility nobody can see tends to shrink in practice. If the only evidence of review is a signature, the organization can’t tell a careful read from a two-second click. Geoff Woods, writing for business leaders, argues that the human has to supply the context and judgment that shape the final output.[5] In a medical record, that judgment ought to leave a trace somebody can audit.
Questions for the next governance meeting
1. Can an auditor, or the next clinician, tell which parts of a signed note were AI-drafted and which were edited by a person?
2. Are reviewers taught to look for the error types the studies describe, like planned versus performed and symptom versus diagnosis?
3. Does anyone compare a sample of signed AI notes against the transcript, and how often? Omissions mostly show up that way.
4. How do patients learn a scribe is listening, and what happens for visits in a language the tool doesn’t support? Permanente’s rollout used a consent template and was limited to English-only encounters at the time.[1]
The time savings are worth chasing. Before the next expansion, ask someone to pull ten signed AI-drafted notes from last week and check them against what was said in the room. Then decide whether your review step is doing the job you’re counting on it to do.
Author Bio
