Challenges in Healthcare IT

Over the next 18 months, healthcare organizations will continue to face:

• Increasing regulatory compliance requirements for CMS and ONC policies for 21st Century Cures Act to securely share patient data across providers and payers
• Pressure to reduce infrastructure costs and move to cloud-based EHR/IT systems
• Shorter deployment cycles demanding rapid, high-quality releases
• Challenges to integrate IoT health devices that require even more IT support

Test automation and test management play a crucial role in mitigating these challenges by reducing manual efforts, ensuring data accuracy in compliance reports, and improving overall traceability.

The Solution

Merito designed a structured framework powered by Tricentis to automate Epic testing along with 250+ application integrations. The solutioned incorporated:

• Just in-time Clarity Test Data Management - Protecting sensitive data in lower-level environments while maintaining integrity.
• Customized Test Management System - Automated alerts to the next test owner with relevant data pointers, eliminating the need for manual research.
• System Integration Testing at a Fraction of the Cost - Drastically reducing resources needed and the time spent waiting in queue for end-to-end validation.
• Incremental Rollout Approach - Avoiding disruption by integrating changes gradually and collecting iterative feedback through qTest.

Results & Impact

By automating Epic and its integrated apps, the organization saw measurable improvements:

• Test Case Count - Manual test case reduction by 75% with similar coverage
• Workflow Optimization - 64% improvement in process efficiency across departments
• Defect Identification - 58% faster issue resolution, improving overall release health
• Cost Savings - Over $880k annually saved in testing and operational efficiencies
• Scalability - A robust framework that extended automation outside Epic to other mission critical applications

Manual Test Cases Reduced by 75%

Before:

• Initial Test Case Executions per year: 1,000 manual test executions
• Every system update or Epic upgrade required manual end-to-end testing across 250+ integrated applications
• Manual testing involved test script execution, data verification, and defect documentation, requiring cross-team coordination and A LOT of effort

With Automation:

• Automated regression tests covered large parts of Epic and downstream applications leaving human intervention for edge cases
• Automation was also used for data creation and cleansing, not just testing
• Test execution happened in hours instead of weeks

Impact:

Before automation, a full manual test cycle took 4-6 weeks per release. With automation, testers could focus on edge cases and let automation do a bulk of the work. The cycles completed in under 10 days, allowing for faster software releases with translated into lower costs and improved reliability. Automated data cleansing tasks were invaluable for tasks like discharging a patient and cleaning 32 rooms in PCU, enabling a transfer of a patient accordingly.

Test Workflow Optimization Improved 64%

Before:

• Testing happened in War Rooms with 30 people sitting idle
• Manual test data creation, pre-cycle
• Tracking Test ownership and execution manually (Excel)
• Waiting for approvals before proceeding to the next test

With qTest to structure test management:

• Test data was pre-generated and dynamically assigned, removing delays caused by test engineers manually finding relevant patient records
• Epic and downstream application workflows were validated concurrently, rather than sequentially, increasing efficiency
• qTest ensured proper execution handoff, allowing teams to trigger tests hooks, receive alerts, validate and document results efficiently
• Manual test cases embraced module approach for better reusability

Impact:

Previously, the customer would spend 3-4 weeks preparing for testing and 2 full weeks of testing due to dependencies and delays. Now, with automation, multiple test scenarios could run in parallel, completing in minutes*, significantly optimizing the overall workflow. It is estimated 64% in measured improvements for overall testing processes, coordination, and data management.

*Some 2 to 5-day E2E scripts invovled batch processing and testing would trigger next day through Distributed Execution.

Defect Identification Improved by 58%

Before:

• Initial Defect Resolution Time was ~48 hours

Pain points:

In manual testing, defects were often detected late in the process through manual test, requiring multiple iterations before release:

• Testers manually logged defects, often missing key reproduction steps
• Fixing and retesting defects sometimes required re-execution of entire test cases rather than targeted validations
• Data inconsistencies across lower-level environments led to false positives, consuming unnecessary debugging time

With Tosca automation and qTest:

• Automated test execution captured detailed logs and screenshots, allowing developers to diagnose and resolve issues faster
• Defect triage became more efficient, as the customized test management system automatically flagged defects and assigned them to the right team and included pertinent information for resolving the defects
• Resolution times decreased from ~48 hours to less than 1 day (20 hours)

Impact:

Previously, defect identification and resolution would take an average 48 days per issue, leading to release delays. With automated testing and structured defect tracking, critical defects were identified and resolved within hours, significantly improving release confidence.

Cost Savings: $880,000 Annually

Before:

• Initial Upgrade/Deployment Costs were $6.7M which covered labor costs, infrastructure, defect remediation, and lost operational hours due to inefficient testing and rework.

Pain points:

Before revamping SDLC, the hospital spent millions on manual testing efforts, including:

1. Extensive labor costs for QA engineers, business analysts, and IT support staff manually executed test cases
2. Delayed releases without automation meant releases took longer and therefore increased operational costs
3. Defect remediation for bugs found late in testing required rework from multiple teams
4. Regulatory compliance overhead required more effort was needed for audits, documentation and traceability

With automation and Test Management:

• Regression testing time dropped resulting in the need for fewer testers
• Faster releases meant fewer delays and impact to the business, reducing system downtime and revenue loss
• Defects were caught earlier reducing compliance without additional manual tracking costs

Impact:

Previously, each Epic upgrade and other major system changes required a six-week manual testing window, where multiple teams worked around the clock away from their daily tasks. After SDLC enhancements were fully implemented, these same activity costs were reduced by $800,000, significantly cutting down the cost of labor and system downtime and positively impacting annual budget constraints.  

Scalability: Priceless

Before:

• Testing was mundane, dreaded, and was seen as a painful initiative

Pain points:

• Testing was limited to a fraction of Epic and downstream applications due to time constraints
• Teams focused on high-risk areas while leaving secondary workflows untested
• Scalability was bottlenecked by manual test execution

With Automation and Test Management:

• More test cases were executed per cycle (parallel execution, 24/7 test runs)
• Automated testing covered Epic workflows plus 250+ integrated applications (for example Sunquest, Pyxis, Experian, CommonWell, My Chart, etc.)
• More environments were tested (QA, Cert B, Cert A, Cert Pre-prod), ensuring significant validation before go-live

Impact:

Before enhancing the SDLC tools, it was estimated only 40 to 45% of the hospital’s workflows were tested each release simply due to time constraints. With automation and modularizing manual test cases, it was estimated to have jumped to over 80%, improving confidence and reducing production issues. Testing more with less effort, meant fewer missed defects, reducing post-release firefighting and improving system reliability.

Conclusion

The implementation of test automation and test management tools at this healthcare provider fundamentally changed how system reliability and performance were managed. With Tricentis Tosca and a structured test management framework, testing became faster, more scalable, and highly effective at ensuring quality across Epic EHR and their 250+ integrated applications.

Test cases were reduced by 75%, eliminating lengthy manual testing delays and accelerating release validation. This resulted in over $800,000 in annual cost savings, primarily by reducing labor costs, infrastructure overhead, and error remediation. Workflow efficiency improved by 64%, allowing IT and clinical teams to collaborate more effectively and significantly streamlining the testing lifecycle.

By enhancing defect identification and resolution by 58%, the organization minimized production issues and ensured system stability with quicker response times. Test coverage expanded by 80%, validating more workflows, integrations, and compliance requirements across the hospital ecosystem.

These advancements created a measurable impact on operational efficiency and patient care. Automated testing ensured that critical healthcare systems remained stable and accurate, allowing medical professionals to focus on providing exceptional patient care without system interruptions. As regulatory demands and healthcare technology continue to evolve, this case study demonstrates how modernizing the Software Development Lifecycle is a necessary investment in the future of healthcare IT.