AI simulation is now a budget-line decision, not a pilot curiosity. Springfield Technical Community College (STCC) has made that call twice: it adopted Spark virtual patients in 2023 and added the SimVox AI voice platform in 2025.
STCC's SIMS Medical Center is accredited by the Society for Simulation in Healthcare, a rare distinction for a community college. It supports 16,000 to 20,000 simulated student experiences a year across 26 degree and certificate programs. In a PCS.ai fireside chat during Healthcare Simulation Week 2026, SIMS Coordinators Dan Taibbi and Daniel O'Neill shared exactly how they evaluate and test AI simulation platforms.
The short answer: "If it's not benefiting their educational experience, then what are we doing?" For STCC, that means customizable cases, consistent AI, and a partner who shows up. They also look for realistic and diverse patients, ease of use, and a future-focused roadmap. They pilot small, test the AI for replicability, and use virtual sim as a stepping stone to high-fidelity simulation, not a replacement.
STCC evaluates every AI simulation platform against one North Star: does it measurably improve student education? "The buzzwords are cool and everything, the tech is impressive, but if it's not benefiting their educational experience, then what are we doing?" said Dan Taibbi. From there, the team named seven criteria:
Customization was the linchpin because STCC builds every sim, virtual or in-person, from the same faculty scenario form. Before Spark, the team reviewed six to eight VR applications and struggled with custom content. On one platform, they never completed a single custom case because the editor was too complicated.
Spark was the first platform where they felt "almost instantly enabled" to modify cases. STCC now has a part-time team member who takes professors' forms, adapts or builds a case in Spark, and tests it with the professor. That loop is much shorter than they've seen with other products.
A partner shows up when you need backup, takes your feedback, and checks in without being chased. STCC asks four questions of every company:
That last one matters. STCC runs its own network, so it skipped the usual IT approval hurdle. Most colleges won't. O'Neill's advice: choose a company willing to get on a Zoom with your IT department. Other software companies told him that was his problem.
Ask the same AI patient the same question on different days and compare the answers. O'Neill calls it a replicability test and ran it across the IMSH show floor this year. Try "How are you feeling right now?" or "How is this condition affecting you?"
"If I ask how you're doing with your congestive heart failure, and one time you say terrible, and one time you say I'm feeling great," that tells you a lot about the AI. Four different answers to one question should prompt hard questions for the vendor. Taibbi added that Spark is not built to piggyback on someone else's large language model; it is vetted and validated, which reduces hallucination risk.
Conversation matters more. O'Neill, who has a filmmaking and game-engineering background, expected graphics to matter most. Instead, the emotional impact of the conversation left the bigger mark across every VR platform he tried.
His proof is Nisha, a combative, drug-seeking Spark patient. He expected to calm her down with ease and lasted about three and a half minutes before taking off the headset. "It's the conversational AI that really makes or breaks it."
Take a breath and give yourself time to research. Most webinar attendees (about 90%) said they had not started, were researching, or were actively evaluating. STCC's advice:
Use these questions in your next demo or committee review.
|
Criterion |
Question to ask the vendor |
What a good answer looks like |
|---|---|---|
|
Educational impact |
How does this improve learning against our objectives? |
Tied to specific learning objectives, not features |
|
Customization |
Can our team build or edit a case from a faculty scenario form? |
Your staff completes a custom case quickly, without vendor help |
|
AI validity |
Is the AI medically validated, or built on a general-purpose LLM? |
Vetted, validated model with guardrails against hallucination |
|
Consistency |
Will two students get the same clinical story from the same case? |
Same answer to "How are you feeling?" across days |
|
Realism and range |
How many patients, backgrounds and languages are available? |
Wide library that looks, acts and feels real |
|
Ease of use |
How fast can staff, faculty and students get onboarded? |
Minutes, for every tech comfort level |
|
Partnership |
Will you join our IT and security review? How is feedback used? |
Yes, with named people, proactive check-ins |
|
Future focus |
What's on the roadmap as AI models change? |
Regular upgrades and new capabilities |
|
Flexibility |
Where can students use it? |
Home, lab, group, one-on-one and high-fidelity rooms |
What AI simulation platforms does STCC use? STCC uses Spark, PCS.ai's screen-based and VR virtual patient platform, adopted in 2023. It also uses SimVox, PCS.ai's AI voice platform for manikin-based simulation, added in 2025.
How many platforms did STCC review before choosing Spark? STCC reviewed six to eight VR applications and several AI-only tools before choosing Spark.
What's the simplest test of an AI patient's quality? Ask the same patient the same question on different days. Inconsistent answers mean the AI can't support fair grading.
Every program's evaluation looks a little different. Book a tailored walkthrough to see how Spark and SimVox fit your curriculum, simulation goals and faculty workflows. Want the full conversation? Watch the webinar recording.
Curious how STCC rolled it out? Read How STCC Uses AI Simulation Across Its Curriculum.