What if an AI-powered virtual patient could do more than simply generate conversational responses?
What if an AI-powered virtual patient could actively interpret not just what learners ask, but how they ask it—assessing tone, empathy, and intent—and then deliver formative feedback comparable to experienced faculty or Standardized Patients?
This is the trajectory we’ve pursued at PCS.ai: advancing from conversational simulation to a feedback-driven AI capable of supporting clinical communication skill development at scale.
And in our recent Virtual Learning Lab, “From Conversation Partner to Communication Coach,” founder and CEO, Balazs Moldovanyi and I shared how PCS has transformed our AI from a simple conversational tool into a true clinical communication coach—one capable of delivering the kind of feedback we’ve known learners need, but educators haven’t always had the time or resources to provide. The challenge has never been about commitment—it’s been about scale.
Why Communication Matters
At PCS, we’ve always focused on communication. It’s in our name—Patient Communication Simulators—and it’s at the heart of safe, effective, and compassionate care.
The research is clear: strong communication isn’t just a “nice to have.” It’s a predictor of real-world performance, a core clinical competency, and a patient safety imperative.
Yet, despite its importance, communication training has long been limited by resource constraints. Standardized Patients (SPs) and faculty provide incredible value—but they face barriers in consistency, availability, and scale. The average learner might only receive about 65 minutes of verbal feedback per year.
Today's advanced AI technologies are uniquely positioned to overcome these challenges. AI-driven solutions like PCS.ai can offer consistent, scalable, and readily available communication training and feedback, significantly enhancing the frequency and depth of learning interactions. By leveraging AI, we can ensure that learners receive immediate, personalized feedback whenever they practice, bridging gaps in traditional methods.
How PCS.ai Uses AI to Deliver Both Qualitative and Quantitative Feedback
Our Approach to Qualitative Feedback
At PCS.ai, we deliver this nuanced, formative feedback through a carefully engineered two-layer approach: a foundational System Prompt combined with Scenario-specific Prompt.
The System Prompt serves as the AI’s compass, setting consistent standards across all scenarios. It tells the model how to behave, what to prioritize, and how to interpret and respond to learners. It also establishes practical constraints, such as the ideal length of feedback and procedures for handling incomplete data. By keeping this foundational prompt standardized and behind-the-scenes, PCS.ai ensures dependable, uniform feedback quality across the platform.
The Scenario Prompt functions as the AI’s detailed map, allowing precise tailoring of feedback to meet specific learning objectives. Scenario authors have the flexibility to emphasize particular competencies—like rapport-building, shared decision-making, or effectively closing a patient encounter. For convenience and consistency, PCS.ai provides professionally crafted default prompts, ensuring high-quality feedback right out of the box.
This dual-layered approach generates deeply insightful feedback. PCS.ai doesn’t merely note whether certain actions were completed; it assesses the quality of learner engagement. Did learners establish trust effectively? Was their language clear and supportive of patient understanding? Did they exhibit empathy during critical moments? By analyzing the entire encounter, the AI delivers rich, example-driven insights similar to those provided by expert faculty or trained standardized patients.
This detailed, narrative-driven feedback moves learners beyond basic checklist criteria. It encourages self-reflection, enhances critical communication skills, and fosters continuous professional growth—growth that ultimately translates into improved patient care.
Why Quantitative Feedback Is So Hard to Get Right
While qualitative feedback is about nuance, tone, and conversation quality, quantitative feedback is about precision. Did the learner ask the right questions? Did they gather the correct information? And did they do it in a way that aligns with the clinical objectives of the scenario?
That sounds straightforward—until you realize that learners will challenge their scores. They want to know exactly why they received or didn’t receive credit. And when every point counts, the system must be accurate, consistent, and defensible—every single time.
PCS.ai’s early approach relied on classification-based AI, using thousands of conversation topics and associated responses to match learner utterances and assign credit. But these rules-based systems struggled as scenarios became more natural, open-ended, and context-driven.
From Categories to Context
When large language models (LLMs) became viable, everything changed. Conversations flowed more naturally, but they also became harder to score using predefined categories. Simple ‘if-this-then-that’ logic was no longer enough.
We needed a new approach—one that could evaluate the entire transcript, understand intent, and determine whether specific clinical concepts were surfaced in context.
To do this, we developed a method we call Conversation Criteria—pattern-matching logic that evaluates whether key topics were meaningfully addressed. But even this system, while necessary, was just a stopgap.
Solving the Quantitative Challenge—Definitively
To deliver scalable, real-time, and scenario-specific scoring, we needed more than clever logic—we needed data. Lots of it. And we needed to use it smartly.
So we did three things:
- Manually validated data: We hand-reviewed simulation transcripts, defining credit criteria and verifying AI scoring outputs to ensure consistency and accuracy.
- Large-scale evaluation using commercial LLMs: We used powerful commercial models—like OpenAI's—to analyze thousands of additional sessions, generating high-quality training data (even though those models were too slow and expensive for live scoring).
- Training our own production-ready model: With that data, we trained a mid-sized, custom LLM that can deliver real-time, cost-effective scoring across our entire Simulation Cloud.
The result? A feedback engine capable of scoring learners with a level of accuracy that rivals trained observers—and that continues to improve as we add more scenarios, refine our criteria, and scale our data pipeline.
I'm still learning the platform, but I've already been able to replicate a key second-year assessment that typically requires a live standardized patient and hours of faculty grading. The AI grades using our rubric with impressive accuracy—very close to what a faculty preceptor would give. —MD Faculty, Florida State University
At Scale and With Purpose
At PCS.ai, our mission has always been clear: to make meaningful, high-quality clinical communication training more accessible, consistent, and scalable. By combining qualitative nuance with quantitative rigor, we’ve built an AI-powered feedback engine that doesn’t just simulate conversation—it actively supports learner growth. As we continue to evolve, our focus remains on empowering educators and elevating the learner experience—because when learners receive the feedback they need, when they need it, better communication—and better care—follows.
