Every Epic Upgrade Is a Bet on Your Revenue Cycle. Test Automation Is How You Stop Gambling.
It's been a few months since the HIMSS Conference in April which always has a ton of great information. If you weren't able to attend, be sure to pencil it in for next year. If you were there, you probably heard the pitch from at least one vendor saying "buy a test automation tool, point it at Epic, and your testing problem goes away." And if you've talked with the Merito team, you've probably heard us say, "that sounds great, but it's wrong."
The tool is the easy part. It's the cheapest line item and potentially, in the grand scheme of things, the least important decision you'll make in the whole effort. Don't take that out of context…the tool is incredibly important… but there are a few things that are so much more important. What actually determines success of your automated testing protect is the stuff no presales engineer puts in a demo. They leave out the validation model behind the scripts, the environment they run in, the release discipline that fires them at the right time, and whether you can do any of it without putting real patient data at risk.
I'm writing this primarily to get the point across that Epic test automation succeeds or fails on framework execution, not necessarily the automation tool itself. And before we dive into it, one hard rule… do not automate Epic through Citrix. I'll come back to why, but take note of it now, because half the vendors in this space will tell you the opposite and they're wrong and it's detrimental to your budget. I'm passionate about this point because my team gets 3-4 calls a year where a healthcare company bought a tool or bought a quality assurance package from a vendor, and somebody calls us to save the project after 12 months of evaporated budget, with little to no progress being made.
Healthcare
EHR
First things first, this is a revenue and safety problem, not an IT project
If you're a director, a CHCIO, or sitting on a board, here's why this lands on your desk and not in the server room. Epic isn't an application you run. It's the operational core of the organization. KLAS Research has Epic holding around 44% of the U.S. acute care hospital market and about 57% of staffed beds. For most of you, Epic is the system. Scheduling, clinical documentation, charge capture, billing, claims, patient access, all of it flows through one platform, and all of it touches money or patient care.
When that platform breaks, is not an "oops" or an inconvenience. It's a business outage. A College of Healthcare Information Management Executives (CHIME) study put the cost of a one-hour EHR outage at about $1.7 million for a medium-sized hospital and up to $3.2 million for a large one. The Ponemon Institute pegs healthcare downtime at roughly $7,500 per minute. Those are what I call the "loud failures" and are the ones that make the news. The quiet ones cost you more, and they're the ones bad testing causes.
A botched Epic update or upgrade rarely takes the whole system down. It breaks a single interface, like a single charge trigger. One registration field that used to be required and now isn't. Nothing crashes and you don't get a page at 2am. No ServiceNow tickets…and that is worse because the failure is silent. Until it surfaces weeks later as denied claims and leaked revenue.
For a moment, consider the climate you're operating in. Experian Health's 2025 State of Claims report found 41% of providers now see more than one in ten claims denied, up from 38% a year earlier, and a great deal of that traces back to preventable errors at registration time. A separate analysis of more than 2,300 hospitals found net revenue leakage jumped roughly 25% in a single year. I don't care what industry you are in, that's a testing problem wearing a finance costume.
Then there's the regulatory side of it. Mishandled PHI and HIPAA civil penalties now run up to about $2.19 million per violation category, per year, under the latest OCR figures. So when we talk about how your automation framework handles patient data, we're talking about real exposure.
What you're actually testing
People who haven't done Epic testing imagine clicking a button and confirming a screen. People who have done it know better.
A serious Epic test isn't a click. It's a multi-day business process. For example, a radiation oncology integration you will want to register a brand-new patient, schedule them across departments with multiple providers, fire ADT events into a third-party treatment planning system, let overnight batch jobs close and reopen accounts on the right days, and then confirm the charges land on the correct hospital account, survive into billing, and make it all the way through coding and claims without falling out. That single scenario spans Prelude, Cadence, Beacon, Grand Central, and Resolute, plus the interfaces between them and the document management feeding scanned consents back into the chart. It takes days to run because the system itself takes days to behave.
That is the reality of Epic QA. And it's exactly why happy-path validation is a trap. The dangerous defects don't live where everything goes right. They live in the seams with the interface message that drops or the account that doesn't close overnight. Same with the charge that posts to the wrong hospital account record or the coverage that should have blocked a claim and quietly didn't. If your testing only proves the system works when every input is perfect, then you've validated a fantasy.
The tool doesn't validate anything. Your model does.
I'm going to make a bold statement, and I hope my favorite vendors don't beat me up for saying it! But this is the part vendors skip, so I'll be blunt about it. Most "test automation" in healthcare is happy-path theater. It confirms the system does what it's supposed to when everything cooperates. Congrats! You've automated the demo. I am here to earn your trust and I want my team to be a long-term trusted advisor for you. Do the hard work, and do it right the first time.
Real value comes from negative testing and control validation. You make the system fail safely, on purpose. Feed it the wrong coverage, hand it a duplicate MRN, send the interface messages out of order, drop a charge with no account attached and then prove the system blocks what it should block and surfaces what it should surface. That's the entire point. Exploit resistance in security has a direct analog in revenue integrity, and it's the same discipline where you assume the bad input will happen, and prove your controls hold. That validation-first, resilient-framework discipline is the entire foundation of how we approach test automation, and it's the difference between a suite that protects revenue and one that just turns green on the demo.
The other piece of this is timing. Epic does a great job with their software development pipelines and they are constantly releasing quarterly version upgrades, special updates, and monthly cumulative content. Every one of those is a change to your core systems, and every one can break something you already validated. So you can't treat testing as a twice-a-year regression sprint before go-live. You treat each Epic update as a release gate opposed to a calendar event. Think of it as a continuous verification. If you start thinking this way, then you know what an upgrade can break before it ever touches production, and shrink the blast radius to the workflows you can prove are still safe.
Why most Epic automation programs stall
I've watched plenty of automation programs sputter out, some of which I am even admittedly been apart of. The pattern is remarkably consistent, and the tool is almost never the reason.
They automate the happy path and call it coverage. The scripts pass, the demo dazzles, and the defects that actually cost money never get exercised. Green is good and the dashboard looks fantastic, but the org is exposed.
They run in environments that don't look like production. Wrong build, stale data, missing or stubbed interfaces. Tests pass in the lab and fail in the real world, which is the worst possible place to find out.
They're built on screen-scraping, usually through Citrix. The automation sees pixels, not application objects. It's isolated by design and that should scream maintenance nightmare! A layout tweak, a font change, or a different screen resolution breaks the whole suite.
They have no release discipline. Epic ships an update, nobody re-runs the suite against it, and the release gate everyone assumed existed simply isn't there. I once heard somebody say, "Epic tests their software so we don't do a lot of testing."
They live as orphan scripts with no test management behind them. This makes traceability so difficult with no ownership, and no defect linkage. When a step fails, nobody can even say what it was supposed to prove. This goes back to "green check mark good, red check mark bad" mentality.
Notice that four of those five are process and infrastructure failures. The tool can't fix any of them.
Set the infrastructure up to win, or skip it
If you're going to invest in Epic automation, set the foundation up correctly or don't bother. It might sound daunting, but if you build it right it can apply to other areas of the business. Like Ransomware mitigation, but thats another blog post for another time.
I am saying you need dedicated, production-like lower environments. The right build, representative data, and live or properly simulated interfaces. If your test environment doesn't mirror production then you're testing a delta and shipping the difference. You need stable, governed test data that never includes real PHI. This includes synthetic or masked patients, coverage, and accounts, built to exercise the edge cases you actually care about. You need object-level access to the application so your automation has clean, durable hooks instead of guessing at coordinates. And you need version-controlled test assets wired into a delivery pipeline that runs your suite on Epic's update cadence.
There are hosted configurations that are built to support automation properly, and there are workarounds that aren't. As you onboard Merito into your environment we work with Epic on your behalf to fight some of those battles. It can take a little back and forth with Epic directly, but eventually we end up with configured hosted option that Epic actually supports, not a bolt-on that collapses at the next upgrade. Otherwise, if you are self-hosted then its fairly easy to get infra locked in.
Stop automating Epic through Citrix
I told you I'd come back to this. Here it is in more depth. Plenty of vendors will assure you they can automate Epic through a Citrix-published session. Technically, they can…but just because you can, doesn't mean you should.
When you automate a published or virtualized session, your tool isn't looking at the native application. It's looking at a remote image of it. You lose object recognition which is done by looking at exposed properties from the developers. Every window or control you see on the screen has several of these, and we use a combination of them to identify objects uniquely. When your Citrix session isolates users from these, then you fall back to image matching and screen coordinates, which is the most fragile automation that has ever existed ever! A new button appears, a render runs a half-second slow, or someone changes the resolution…then your suite falls apart. This is what causes your team to spend more time babysitting broken automation than they ever spent testing by hand. That's not a release gate.
The better path is object-level access through locally installed or properly hosted Epic configurations, set up so automation reads the application directly and survives the constant change. It's faster, it's dramatically more stable, and because it's done in a configuration designed for automation, it doesn't break every time Epic improves the product. The vendors steering you toward Citrix automation are optimizing for "we can start tomorrow." You should be optimizing for "this still works after the next four upgrades."
RPA is not a testing strategy
Now the other tooling conversation, because I guarantee someone on your team has already suggested it...What about RPA? You've got UiPath, Blue Prism, Automation Anywhere or Power Automate sitting around, they can drive a user interface, so why not point them at Epic?
Let's be fair before we bash them too hard. RPA tools are good at what they were built for, which is running a repetitive business process the same way every single time. That's a real and useful job. Test automation is not that job.
RPA is built to do a task, not to validate one. It assumes the happy path because the happy path is the point. It doesn't think in assertions, expected-versus-actual, negative cases, or coverage against requirements. It has no native concept of a defect, a test cycle, or traceability back to what you were supposed to verify. Bolt some reporting on top and you've built a brittle robot that confirms the happy path and tells you nothing about what's exposed. In a system where the expensive failures hide in the negative cases, that's another good way to burn budget. Apologies if that's brutal, but again I'm here to save you time and money.
A purpose-built quality assurance platform is the right play, because it was designed around validation from the ground up. Here's the gap:
Real assertions and negative testing, so you can prove the system blocks bad data, not just that it processes good data.
Test data management built for synthetic or masked, PHI-free patients and accounts, so you can exercise edge cases without ever touching a real record.
Defect tracking and requirements traceability, so every failed step maps back to exactly what it was meant to verify.
Reporting built for risk and coverage, not just pass/fail counts that mean nothing to leadership.
A maintenance model that survives Epic's update cadence instead of shattering on every change.
One caveat, because it goes against the whole tool-shopping instinct. Even the best QA platform is just leverage. Execution, enforcement, and process are what make it pay off. A great tool with no validation model and no release discipline is an expensive way to generate some red and green checkmarks. Buy the right category, then put the discipline behind it.
Where AI actually helps, and where it can't
We have a partnership with Anthropic and we use Claude across our automation practice, so let me give you the honest version rather than the hype version.
AI does not replace your validation model, and it does not replace your testers. What it removes is the grind that makes automation expensive to build and brutal to maintain. We use Claude to draft test cases straight from build documentation and requirements. To generate realistic but entirely synthetic test data. To read a change in an Epic update and flag which scripts it's likely to break before you run anything, which is real change-impact analysis instead of find-out-the-hard-way. To repair locators and steps when the application shifts. And to turn a failed run into a few sentences a human can act on, instead of four hundred rows of raw status.
The hard truth is that AI isn't going to do all your testing for you. But it certainly can help and you will need to put some boundaries and guardrails in place so your QA team doesn't risk data loss/exposure issues trying to make it work like that. Our strategies never require real PHI, and none of it should touch real PHI. The pattern that keeps you safe is simple and non-negotiable: real patient data never enters the AI workflow. We generate synthetic patients, synthetic coverage, and synthetic accounts. Inputs are de-identified. The deployment is governed, with enterprise terms, no training on your data, and the logging and controls that hold up to an audit, plus a business associate agreement anywhere protected data could be in scope. AI that's careless with PHI doesn't save you time. It hands you the exact multi-million-dollar penalty exposure you were trying to avoid. When used correctly, its the safest and fastest path.
Automation without test management is just scripts in a folder
We've talked a lot about automation, but that is the engine. Test management is the system of record that makes the engine accountable. It's where your test cases live, where execution status is tracked across cycles, where every step links to a requirement and every failure links to a defect, and where you can see whether your coverage is improving or quietly rotting. Look at any serious Epic integration script and you'll see this in the bones of it: owners, status columns, defect numbers, notes carried across rounds and dates. That governance is the difference between "we have automation" and "we can prove what our automation verified, when, and what we did about every failure it found." Without it, you've got a pile of scripts nobody can vouch for, which in an audit is worse than having nothing. We stand up the test management layer alongside the automation, so you know what your program is doing and can actually talk to the business value that's been created.
Report value in dollars and risk, not test counts
A lot of times boards and executives aren't quite sure what type of reports they want. Please don't report on test case count. Its meaningless. If you are on the board, ask what revenue critical workflows are under continuous coverage, what defects were caught in the Cert environment, right before Go-Live, or what dollars were saved if those defects would have leaked, then compare regression cycle duration to your previous non-automated baseline, how much risk was retired before it ever reached production.
As Quality Assurance people, we want to translate testing into the language leadership already speaks, which is protected revenue, avoided downtime, reduced denial exposure, and defensible compliance. Do that and QA stops looking like a cost center and starts looking like one of the few things actively protecting the P&L. We build that reporting in from the start, because the work isn't real to leadership until it's measured in their terms.
What about Cerner?
This is written for Epic because Epic is where most of our customers live. But the approach applies to EHR in general, so if you're on Oracle Health (Cerner, still the clear number two at roughly 22% of acute care hospitals), don't tune out.
The strategy is identical: negative testing, production-like environments, release gates tied to update cadence, and zero real PHI. What changes is mechanical, not philosophical. Object recognition works differently across the platforms, and the business process flows are laid out differently, so the scripts and the application hooks aren't interchangeable. Integration testing, though, is very nearly the same exercise on either system. HL7 and FHIR interfaces, ADT events, charge and claim flows all behave alike because they're solving the same underlying problem. Get the validation model right and most of it ports. The platform changes the implementation but it doesn't change the thinking.
The gate is coming. Be ready for it.
Your next Epic upgrade is on the calendar whether your test automation is ready or not. The only real question is where you find the broken charge trigger. In a lower environment next week, where it costs you a fix? Or in a denial report next quarter, where it costs you revenue, rework, and a hard conversation with finance?
So ask your team three things, and listen closely to how confidently they answer. Can you prove, right now, which workflows are under automated coverage? Can you deliberately fail the system and watch it block bad data and bad charges? And when Epic ships the next update, does a suite actually run against it before it reaches production, or does someone just hope it's fine? If the honest answer to any of those is "not really," you don't have a tooling problem. You have a validation and release-discipline problem, and that's the one that costs money.
Merito does this two ways, we can teach your team to build and run it, or we can build and run it for you. When you need senior QA and test-automation engineers embedded alongside your people to actually stand it up, that's where our staff augmentation comes in, so the people who do this discipline for a living are in the room from day one. Either way, we set the infrastructure up correctly from the start, we work directly with Epic on hosted configurations so you're not betting your release gate on a Citrix workaround, and we use our Anthropic partnership to make the whole program faster to build and cheaper to maintain without ever putting a single real patient record at risk.
If your next upgrade is scheduled and your validation model isn't, that's the conversation to have now. Let's talk before the gate you're counting on turns out not to be there.
Epic test automation requires a structured validation framework rather than simply purchasing an automation tool. Successful programs combine production-like environments, synthetic test data, object-level application access, release discipline, and strong test management practices. Merito helps healthcare organizations design, implement, and manage Epic automation strategies that validate critical workflows, reduce upgrade risks, protect revenue integrity, and support safer EHR releases through proven healthcare QA expertise.
Hospitals can reduce Epic upgrade risks by implementing continuous verification practices that test revenue-critical, clinical, and integration workflows before changes reach production. Effective Epic automation validates negative scenarios, interface behavior, billing workflows, and patient access processes. Merito partners with healthcare organizations to build automated regression frameworks, establish release gates, and create sustainable QA processes that help identify defects earlier while reducing operational disruption during Epic upgrades.
Epic test automation projects often fail because organizations focus on tools instead of execution strategy. Common challenges include automating only happy paths, using unstable environments, relying on screen-based automation, lacking test governance, and failing to align testing with Epic release cycles. Merito helps organizations avoid these issues by implementing the right automation framework, infrastructure, test management processes, and healthcare-specific QA practices needed for long-term success.
Healthcare organizations should evaluate an Epic test automation partner based on healthcare expertise, implementation experience, automation methodology, test governance capabilities, and understanding of Epic workflows. The right partner should support environment readiness, synthetic data management, integration testing, reporting, and ongoing optimization. Merito provides end-to-end Epic automation services, helping organizations implement, operate, and continuously improve QA programs aligned with business, compliance, and patient safety goals.
AI can improve Epic test automation by accelerating test case creation, generating synthetic test data, identifying potential automation impacts, and improving maintenance efficiency. However, AI workflows must protect sensitive healthcare information through proper governance, de-identification, and security controls. Merito helps organizations adopt responsible AI-enabled automation approaches that improve testing speed while ensuring PHI protection, compliance readiness, and enterprise-grade security practices throughout the QA lifecycle.
Citrix-based Epic automation relies on screen interactions rather than direct application object recognition, making automation fragile and difficult to maintain. Interface changes, resolution differences, or minor application updates can cause failures. A stronger approach uses supported hosted or locally configured environments that allow stable object-level automation. Merito helps healthcare organizations establish reliable Epic automation infrastructure designed to withstand upgrades and reduce long-term maintenance challenges.
RPA focuses on automating repetitive business tasks, while Epic test automation focuses on validating whether healthcare workflows function correctly under different scenarios. Test automation requires assertions, negative testing, defect tracking, coverage measurement, and traceability to requirements. Merito helps organizations select and implement purpose-built QA solutions that validate Epic workflows instead of relying on RPA tools that were not designed for comprehensive software testing.
Merito helps healthcare organizations implement Epic test automation by combining healthcare QA expertise, automation frameworks, test management practices, infrastructure guidance, and ongoing support. Organizations can work with Merito to build internal capabilities or leverage managed automation services. Merito supports implementation, optimization, and renewal of automation programs while helping teams improve release confidence, protect revenue workflows, reduce testing effort, and maintain scalable QA operations.
Keep Reading
Related Blogs
Explore a few more Merito insights that align with the themes in this article.
