Automated Software Testing of Epic to Drive Efficiency and Accuracy
Customer Impact
Automated Software Testing of Epic to Drive Efficiency and Accuracy
When it comes to ensuring reliability and performance of critical systems, particularly with Electronic Health Records (EHR) platforms like Epic and integrations with their downstream applications, Healthcare IT teams face several unique challenges compared to other industries. One major healthcare provider was struggling with inefficient testing processes, zero automation, and a lack of structured reporting.
Merito implemented a comprehensive test automation strategy to enhance testing efficiency and enable a Ready-Set-Go testing approach. This provided key stakeholders with an easy to use, collaborative environment, empowering doctors and nurses with limited IT knowledge, to participate in user acceptance testing (UAT). The initiative established clear communication channels, validation/reporting structures, and ultimately ensured higher quality releases which improved systems for patient care and safety.
Start a DevOps consulting engagement
Build the operating model and roadmap before the next implementation
If your team needs a clearer DevOps strategy, a defensible DevSecOps approach, a rationalized toolchain, or a roadmap before the next implementation, Merito can scope the engagement and start the current-state assessment.
Over the next 18 months, healthcare organizations will continue to face:
Increasing regulatory compliance requirements for CMS and ONC policies for 21st Century Cures Act to securely share patient data across providers and payers
Pressure to reduce infrastructure costs and move to cloud-based EHR/IT systems
Challenges to integrate IoT health devices that require even more IT support
Test automation and test management play a crucial role in mitigating these challenges by reducing manual efforts, ensuring data accuracy in compliance reports, and improving overall traceability.
The Solution
Merito designed a structured framework powered by Tricentis to automate Epic testing along with 250+ application integrations. The solutioned incorporated:
Just in-time Clarity Test Data Management - Protecting sensitive data in lower-level environments while maintaining integrity.
Customized Test Management System - Automated alerts to the next test owner with relevant data pointers, eliminating the need for manual research.
System Integration Testing at a Fraction of the Cost - Drastically reducing resources needed and the time spent waiting in queue for end-to-end validation.
Incremental Rollout Approach - Avoiding disruption by integrating changes gradually and collecting iterative feedback through qTest.
Results & Impact
By automating Epic and its integrated apps, the organization saw measurable improvements:
Test Case Count - Manual test case reduction by 75% with similar coverage
Workflow Optimization - 64% improvement in process efficiency across departments
Cost Savings - Over $880k annually saved in testing and operational efficiencies
Scalability - A robust framework that extended automation outside Epic to other mission critical applications
Manual Test Cases Reduced by 75%
Before:
Initial Test Case Executions per year: 1,000 manual test executions
Every system update or Epic upgrade required manual end-to-end testing across 250+ integrated applications
Manual testing involved test script execution, data verification, and defect documentation, requiring cross-team coordination and A LOT of effort
With Automation:
Automated regression tests covered large parts of Epic and downstream applications leaving human intervention for edge cases
Automation was also used for data creation and cleansing, not just testing
Test execution happened in hours instead of weeks
Impact:
Before automation, a full manual test cycle took 4-6 weeks per release. With automation, testers could focus on edge cases and let automation do a bulk of the work. The cycles completed in under 10 days, allowing for faster software releases with translated into lower costs and improved reliability. Automated data cleansing tasks were invaluable for tasks like discharging a patient and cleaning 32 rooms in PCU, enabling a transfer of a patient accordingly.
Test Workflow Optimization Improved 64%
Before:
Testing happened in War Rooms with 30 people sitting idle
Manual test data creation, pre-cycle
Tracking Test ownership and execution manually (Excel)
Waiting for approvals before proceeding to the next test
With qTest to structure test management:
Test data was pre-generated and dynamically assigned, removing delays caused by test engineers manually finding relevant patient records
Epic and downstream application workflows were validated concurrently, rather than sequentially, increasing efficiency
qTest ensured proper execution handoff, allowing teams to trigger tests hooks, receive alerts, validate and document results efficiently
Manual test cases embraced module approach for better reusability
Impact:
Previously, the customer would spend 3-4 weeks preparing for testing and 2 full weeks of testing due to dependencies and delays. Now, with automation, multiple test scenarios could run in parallel, completing in minutes*, significantly optimizing the overall workflow. It is estimated 64% in measured improvements for overall testing processes, coordination, and data management.
*Some 2 to 5-day E2E scripts invovled batch processing and testing would trigger next day through Distributed Execution.
Defect Identification Improved by 58%
Before:
Initial Defect Resolution Time was ~48 hours
Pain points:
In manual testing, defects were often detected late in the process through manual test, requiring multiple iterations before release:
Testers manually logged defects, often missing key reproduction steps
Fixing and retesting defects sometimes required re-execution of entire test cases rather than targeted validations
Data inconsistencies across lower-level environments led to false positives, consuming unnecessary debugging time
With Tosca automation and qTest:
Automated test execution captured detailed logs and screenshots, allowing developers to diagnose and resolve issues faster
Defect triage became more efficient, as the customized test management system automatically flagged defects and assigned them to the right team and included pertinent information for resolving the defects
Resolution times decreased from ~48 hours to less than 1 day (20 hours)
Impact:
Previously, defect identification and resolution would take an average 48 days per issue, leading to release delays. With automated testing and structured defect tracking, critical defects were identified and resolved within hours, significantly improving release confidence.
Cost Savings: $880,000 Annually
Before:
Initial Upgrade/Deployment Costs were $6.7M which covered labor costs, infrastructure, defect remediation, and lost operational hours due to inefficient testing and rework.
Pain points:
Before revamping SDLC, the hospital spent millions on manual testing efforts, including:
Extensive labor costs for QA engineers, business analysts, and IT support staff manually executed test cases
Delayed releases without automation meant releases took longer and therefore increased operational costs
Defect remediation for bugs found late in testing required rework from multiple teams
Regulatory compliance overhead required more effort was needed for audits, documentation and traceability
With automation and Test Management:
Regression testing time dropped resulting in the need for fewer testers
Faster releases meant fewer delays and impact to the business, reducing system downtime and revenue loss
Defects were caught earlier reducing compliance without additional manual tracking costs
Impact:
Previously, each Epic upgrade and other major system changes required a six-week manual testing window, where multiple teams worked around the clock away from their daily tasks. After SDLC enhancements were fully implemented, these same activity costs were reduced by $800,000, significantly cutting down the cost of labor and system downtime and positively impacting annual budget constraints.
Scalability: Priceless
Before:
Testing was mundane, dreaded, and was seen as a painful initiative
Pain points:
Testing was limited to a fraction of Epic and downstream applications due to time constraints
Teams focused on high-risk areas while leaving secondary workflows untested
Scalability was bottlenecked by manual test execution
With Automation and Test Management:
More test cases were executed per cycle (parallel execution, 24/7 test runs)
Automated testing covered Epic workflows plus 250+ integrated applications (for example Sunquest, Pyxis, Experian, CommonWell, My Chart, etc.)
More environments were tested (QA, Cert B, Cert A, Cert Pre-prod), ensuring significant validation before go-live
Impact:
Before enhancing the SDLC tools, it was estimated only 40 to 45% of the hospital’s workflows were tested each release simply due to time constraints. With automation and modularizing manual test cases, it was estimated to have jumped to over 80%, improving confidence and reducing production issues. Testing more with less effort, meant fewer missed defects, reducing post-release firefighting and improving system reliability.
Conclusion
The implementation of test automation and test management tools at this healthcare provider fundamentally changed how system reliability and performance were managed. With Tricentis Tosca and a structured test management framework, testing became faster, more scalable, and highly effective at ensuring quality across Epic EHR and their 250+ integrated applications.
Test cases were reduced by 75%, eliminating lengthy manual testing delays and accelerating release validation. This resulted in over $800,000 in annual cost savings, primarily by reducing labor costs, infrastructure overhead, and error remediation. Workflow efficiency improved by 64%, allowing IT and clinical teams to collaborate more effectively and significantly streamlining the testing lifecycle.
By enhancing defect identification and resolution by 58%, the organization minimized production issues and ensured system stability with quicker response times. Test coverage expanded by 80%, validating more workflows, integrations, and compliance requirements across the hospital ecosystem.
These advancements created a measurable impact on operational efficiency and patient care. Automated testing ensured that critical healthcare systems remained stable and accurate, allowing medical professionals to focus on providing exceptional patient care without system interruptions. As regulatory demands and healthcare technology continue to evolve, this case study demonstrates how modernizing the Software Development Lifecycle is a necessary investment in the future of healthcare IT.
Start a DevOps consulting engagement
Build the operating model and roadmap before the next implementation
If your team needs a clearer DevOps strategy, a defensible DevSecOps approach, a rationalized toolchain, or a roadmap before the next implementation, Merito can scope the engagement and start the current-state assessment.
