Depositphotos 212173612 XL

Evaluation of AI in Patient Communications

Category:

Executive Context: Why Healthcare and AI Virtual Agents Are Still at Odds

Healthcare depends on communication more than almost any other industry. Every patient journey—accessing care, coordinating services, managing chronic conditions, navigating billing, or receiving follow-up instructions—requires repeated, accurate, and empathetic interaction between people and systems. These interactions are not optional; they are intrinsic to the mission of saving lives, improving outcomes, and serving the community.

At the same time, healthcare leaders are under relentless pressure to reduce cost, address workforce shortages, and meet rising consumer expectations. Artificial intelligence is frequently positioned as the answer. Yet healthcare and AI—particularly conversational and speech-to-speech technologies—remain fundamentally at odds.

The reason is not that AI lacks promise. It is that healthcare requires trust and empathy as core attributes of service, and many AI implementations have been deployed in ways that undermine both. Poorly implemented virtual agents, robotic voices, transcription errors, and disconnected workflows have led some health systems to pull back after early pilots. This is consistent with industry analysis; in 2025, Gartner found that as many as 85% of AI implementations fail to deliver expected value, often due to overreach, lack of governance, or misalignment with human workflows.

The lesson for healthcare executives is clear: AI is not about replacing conversations yet. It is about supporting human care delivery, reducing friction, and creating space for empathy—not eliminating it. There is ample opportunity for FTE takeout in the coming 12-18 months. But it has to be in alignment with patient segment (e.g. mothers in labor under 40) or business capability (e.g. those in collections are too embarrassed to speak with a live agent).

Trust and Empathy Are the Real Product

In healthcare, trust is not a “soft” metric. It determines whether patients follow instructions, disclose accurate information, answer outreach calls, or stay within a system for ongoing care. Empathy is not a feature; it is the lens through which every interaction is judged.

Early failures of speech-to-speech and virtual agent deployments illustrate what happens when technology leads and trust follows. Transcription errors—particularly with medication names, dosages, or dates—create immediate safety concerns. Acoustic challenges such as background noise, accents, emotional distress, or weak phone connections degrade accuracy. Robotic or unnatural synthesized voices damage credibility, causing patients to disengage or assume calls are scams. Latency and poor turn-taking interrupt natural conversation and create frustration. And perhaps most critically, weak integration with EHRs and hospital systems means the AI can “talk” but cannot act, forcing patients to repeat themselves or be transferred anyway.

These failures are not edge cases. They reflect a fundamental truth: healthcare conversations are high-stakes, emotionally charged, and context-dependent. Any AI strategy that does not start with this reality will fail.

Why Speech-to-Speech Must Be Deployed Strategically—Not Universally

Speech-to-speech AI is often presented as the pinnacle of conversational automation. In healthcare, it is also the highest-risk modality. Voice interactions are immediate, synchronous, and emotionally loaded. Errors are felt instantly, and recovery of trust is difficult. In many cases, this technology is not ready. There are some edge cases and solutions that are stand-alone and highly optimized for the task of communications with patients. However, the investment at this time for the technology needs to be thought through.

This does not mean speech-to-speech should be avoided. It means it must be deployed surgically, in places where it augments trust rather than replaces it. The most successful organizations treat speech AI as infrastructure, not a front-line caregiver. They prioritize reliability, latency, integration, and voice quality over novelty.

Strategically deployed, speech-to-speech enables faster access, smoother handoffs, and better agent support. Deployed indiscriminately, it becomes a reputational and operational liability.

A Conservative View: Where AI Can Deliver Real Benefits Today

Contact Center operators are very good at what they do today. They manage a heavy load of inbound and outbound calls, and those calls require a very high touch. They are the ones who know the voice of the patient. In trusting of their teams, healthcare executives should focus on three categories where AI is already delivering measurable value—without asking it to be empathetic on its own.

Live Agent Support: Making Humans Better, Faster, and More Consistent

The most immediate and lowest-risk value of AI is behind the scenes, supporting human agents rather than replacing them.

Natural language understanding (NLU) can classify intent and surface relevant context before an agent joins the call, reducing average handle time without affecting empathy. Real-time transcription—used as an assistive tool, not a system of record—allows agents to stay present in the conversation instead of taking notes. AI-generated call summaries can complete documentation in minutes instead of hours, dramatically reducing after-call work and burnout.

AI-enabled quality evaluations that once took analysts days to compile (and usually using a spot-check method) can now be completed in minutes, improving consistency and coaching velocity. The AI-enabled evaluations, if built correctly, can complete a review of 100% of the calls and score the evaluations, and provide trends to supervisors today. It’s an exceptional and surprising benefit to agents and their managers. Also, real-time knowledge base lookup and retrieval-augmented generation (RAG) services surface accurate, approved answers during the call, reducing misinformation and unnecessary transfers. This works well when listening to callers and helping to provide a link to a document in real-time, preventing minutes of look-up time as well as the expense of placing a patient on hold.

Using these services in these examples, AI should rarely speak for the organization. It empowers the people who do.

Self-Service for Low-Risk, High-Volume Tasks

Where trust is already established and the task is transactional, self-service works. Appointment scheduling and rescheduling, confirmation, reminders, location and preparation instructions, and basic administrative updates are ideal candidates.

These interactions are predictable, measurable, and reversible. When implemented with clear escalation paths to humans, they reduce call volume and free staff capacity

without damaging patient trust. Importantly, success here depends less on conversational sophistication and more on integration quality—the AI must actually complete the task in the system of record.

This is where many early efforts failed: not because patients rejected automation, but because automation could not execute. Today, patients can’t be reached during their working hours but now they will respond generally to two-way text to schedule or reschedule an appointment.

The LLM Front Door: Orchestration, Not Conversation

The emerging “LLM front door” should not be misunderstood as a talking chatbot. Its real value is orchestration—understanding intent, maintaining context, invoking the right workflows, and deciding when a human is required.

In this role, the LLM acts as a traffic controller rather than a conversationalist. It routes patients to the appropriate channel, prepares agents with context, and ensures continuity across voice, SMS, chat, and portal interactions. Over time, it improves access efficiency and reduces friction without asking patients to trust it with empathy.

This approach aligns with healthcare’s reality: empathy is delivered by humans; efficiency is delivered by systems.

Build Trust First, Then Scale Intelligence

The most successful AI strategies in healthcare share common characteristics. They start with trust, not automation. They prioritize integration and reliability over novelty. They deploy speech-to-speech selectively, not universally. And they measure success in operational and human terms—handle time, documentation burden, staff satisfaction, and patient follow-through—not just containment rates.

AI is not yet a caregiver. But it is already a powerful care enabler.

Healthcare does not need more talking machines. It needs better-supported humans, fewer unnecessary interactions, and communication systems that respect the emotional and clinical weight of every conversation.

The real benefits of AI-driven contact centers today are found not in replacing conversations, but in removing friction, compressing administrative work, and creating space for empathy. Speech-to-speech will play a significant role—but only when deployed strategically, governed carefully, and integrated deeply into the fabric of care delivery.

For boards and executive teams, the mandate is clear: invest in AI where it strengthens trust today, and build the foundation that will allow more conversational experiences tomorrow—when the technology, governance, and patient expectations are truly ready.

Written by:

Jeff Hartweg
Vice President
  
Sharad Singh
Associate Director