AI in the Exam Room
The AI Listening in the Room
I was mid-sentence with the doctor when I noticed the small disclosure on the wall. An AI scribe was listening, and I could opt out if I wanted to. To be clear, this was an opt out, not opt in.
As someone who works on healthcare products, I understand logically why ambient AI scribes are rapidly being adopted. They reduce physician documentation burden, allow clinicians to focus more on patients, and can improve workflow efficiency. From a product perspective, the value proposition is compelling.
However, experiencing an AI scribe as a patient was very different. Rather than being asked for consent, I was effectively asked to opt out after the technology had already been introduced into the encounter. That subtle difference fundamentally changed the experience. Instead of feeling like I had agency over how my medical information was handled, I felt I had to actively decline something involving highly personal conversations, which started my interaction with that visit on a negative note. Often, in the early stages of a visit with a provider, a patient is deciding how much to divulge and developing a comfort with their provider, which is often critical to getting a full clinical picture. This becomes an even larger issue if the patient is someone from a demographic that may be entering a clinical setting with a sense of distrust due to past experiences. All of this makes it less likely for the patient to be forthcoming, and more likely for them to walk away with a negative impression, having started the appointment off on the wrong foot.
The experience also raised a broader question: will patients who know they’re being recorded and processed by AI begin to self-censor? If they do, clinicians may receive less complete or less accurate information, potentially reducing the quality of clinical documentation and, ultimately, patient care. While AI scribes are designed to improve data capture, they could unintentionally decrease the quality of the underlying data if patients withhold sensitive information, causing an overall negative interaction.
The Argument for AI Scribes
There are some very good arguments for why AI scribes can be helpful in a clinical setting. In an environment where physicians are already dealing with more demand than they have time for, AI scribes can reduce physician burnout by streamlining documentation and cutting the time spent writing clinical notes. One study of 263 clinicians found burnout decreased from 51.9% to 38.8% among clinicians using AI scribes (JAMA Network Open, “Ambient AI Scribes to Reduce Administrative Burden and Professional Burnout”). That’s not an efficiency metric. That’s the difference between a physician who has enough left to truly be present with the patients they see throughout the day.
In another small study, 69.2% of surveyed patients felt physicians focused on them more when an AI scribe was used (JMIR Medical Informatics, “Impact of an Ambient AI Scribe Among Clinicians and Patients”). This is key for getting a patient to open up and provide all the information a clinician needs to provide treatment.
It can also be beneficial on the insurance side, since it can standardize documentation when it comes time to request authorization.
Cons of AI Scribes
To go back to the example of my experience, opt out can feel fundamentally different from opt in. It requires a patient to be willing and able to advocate for themselves, which not everyone is and can leave them in a situation they never agreed to. Some patients may even feel pressured to accept simply because declining feels awkward. A patient may also not fully understand what is being recorded, and educating the patient on this takes away from the reason for the visit, if it happens at all. There are also examples where consent conversations are inconsistent between clinicians, or even depending on how busy a clinician is that day. One study found 81.6% of patients consented when given only basic information; however, consent dropped to 55.3% when patients were also informed about AI capabilities, data storage, and vendor involvement (JAMA Network Open, “Consent for Ambient Documentation Using Generative AI in Ambulatory Care”).
There’s also the element of trust. Introducing an AI into a personal healthcare interaction can erode trust if not handled transparently. A clinical visit should be a safe place for a patient. It is an environment where sensitive discussions are being had that may involve mental health, sexual health, substance abuse, reproductive health, and more. A patient may not know where recordings are processed, how they’re stored, who has access, or how long the data is retained, all of which can vary depending on the facility.
Self-Censorship
All of this points to the biggest risk: self-censorship. If a patient is not comfortable, they may avoid discussion of embarrassing symptoms, trauma, family issues, mental health concerns, sexual behavior, etc that really should be part of the visit, simply because they know AI is listening. The resulting medical record may actually become less complete, undermining the primary goal of the visit.
Self-censorship can also happen if a patient is unsure how an AI scribe might interpret something. Take the example of past self-harm. The AI might flag that as a current concern even though it isn’t one, and now it’s part of a much larger record, potentially available to every other provider a patient sees, which could influence future visits.
The Core Product Tension
From a product design standpoint, there is one thing I keep coming back to. Is there a version of consent that doesn’t ask a patient to make a privacy decision in the ninety seconds before they’re supposed to be talking about their health? Is there enough time in a clinic visit to properly educate a patient on what opt in means, given that it should be opt in, not opt out?
The interesting product challenge isn’t whether AI scribes improve clinician workflows, because they increasingly do. The harder question is how we optimize clinician efficiency without creating enough discomfort that patients change what they disclose. If ambient AI changes patient behavior, it may inadvertently reduce the quality of the very clinical data it is intended to improve.
I don’t have a clean answer for this one. I just know that the next time I’m in an exam room, I’ll be watching for that consent notice and wondering whether the person before me even noticed it was there.
Articles Referenced:
JMIR Medical Informatics – Impact of an Ambient AI Scribe Among Clinicians and Patients
JAMA Network Open – Consent for Ambient Documentation Using Generative AI in Ambulatory Care
JAMA Network Open – Ambient AI Scribes to Reduce Administrative Burden and Professional Burnout