With Automation:

• Automated regression tests covered large parts of Epic and downstream applications leaving human intervention for edge cases
• Automation was also used for data creation and cleansing, not just testing
• Test execution happened in hours instead of weeks

Impact:

Before automation, a full manual test cycle took 4-6 weeks per release. With automation, testers could focus on edge cases and let automation do a bulk of the work. The cycles completed in under 10 days, allowing for faster software releases with translated into lower costs and improved reliability. Automated data cleansing tasks were invaluable for tasks like discharging a patient and cleaning 32 rooms in PCU, enabling a transfer of a patient accordingly.

Test Workflow Optimization Improved 64%

Before:

• Testing happened in War Rooms with 30 people sitting idle
• Manual test data creation, pre-cycle
• Tracking Test ownership and execution manually (Excel)
• Waiting for approvals before proceeding to the next test

With qTest to structure test management:

• Test data was pre-generated and dynamically assigned, removing delays caused by test engineers manually finding relevant patient records
• Epic and downstream application workflows were validated concurrently, rather than sequentially, increasing efficiency
• qTest ensured proper execution handoff, allowing teams to trigger tests hooks, receive alerts, validate and document results efficiently
• Manual test cases embraced module approach for better reusability

Impact:

Previously, the customer would spend 3-4 weeks preparing for testing and 2 full weeks of testing due to dependencies and delays. Now, with automation, multiple test scenarios could run in parallel, completing in minutes*, significantly optimizing the overall workflow. It is estimated 64% in measured improvements for overall testing processes, coordination, and data management.

*Some 2 to 5-day E2E scripts invovled batch processing and testing would trigger next day through Distributed Execution.

Defect Identification Improved by 58%

Before:

• Initial Defect Resolution Time was ~48 hours

Pain points:

In manual testing, defects were often detected late in the process through manual test, requiring multiple iterations before release:

• Testers manually logged defects, often missing key reproduction steps
• Fixing and retesting defects sometimes required re-execution of entire test cases rather than targeted validations
• Data inconsistencies across lower-level environments led to false positives, consuming unnecessary debugging time

With Tosca automation and qTest:

• Automated test execution captured detailed logs and screenshots, allowing developers to diagnose and resolve issues faster
• Defect triage became more efficient, as the customized test management system automatically flagged defects and assigned them to the right team and included pertinent information for resolving the defects
• Resolution times decreased from ~48 hours to less than 1 day (20 hours)

Impact:

Previously, defect identification and resolution would take an average 48 days per issue, leading to release delays. With automated testing and structured defect tracking, critical defects were identified and resolved within hours, significantly improving release confidence.

Cost Savings: $880,000 Annually

Before:

• Initial Upgrade/Deployment Costs were $6.7M which covered labor costs, infrastructure, defect remediation, and lost operational hours due to inefficient testing and rework.

Pain points:

Before revamping SDLC, the hospital spent millions on manual testing efforts, including:

1. Extensive labor costs for QA engineers, business analysts, and IT support staff manually executed test cases
2. Delayed releases without automation meant releases took longer and therefore increased operational costs
3. Defect remediation for bugs found late in testing required rework from multiple teams
4. Regulatory compliance overhead required more effort was needed for audits, documentation and traceability

With automation and Test Management:

• Regression testing time dropped resulting in the need for fewer testers
• Faster releases meant fewer delays and impact to the business, reducing system downtime and revenue loss
• Defects were caught earlier reducing compliance without additional manual tracking costs

Impact:

Previously, each Epic upgrade and other major system changes required a six-week manual testing window, where multiple teams worked around the clock away from their daily tasks. After SDLC enhancements were fully implemented, these same activity costs were reduced by $800,000, significantly cutting down the cost of labor and system downtime and positively impacting annual budget constraints.  

Scalability: Priceless

Before:

• Testing was mundane, dreaded, and was seen as a painful initiative

Pain points:

• Testing was limited to a fraction of Epic and downstream applications due to time constraints
• Teams focused on high-risk areas while leaving secondary workflows untested
• Scalability was bottlenecked by manual test execution

With Automation and Test Management:

• More test cases were executed per cycle (parallel execution, 24/7 test runs)
• Automated testing covered Epic workflows plus 250+ integrated applications (for example Sunquest, Pyxis, Experian, CommonWell, My Chart, etc.)
• More environments were tested (QA, Cert B, Cert A, Cert Pre-prod), ensuring significant validation before go-live

Impact:

Before enhancing the SDLC tools, it was estimated only 40 to 45% of the hospital’s workflows were tested each release simply due to time constraints. With automation and modularizing manual test cases, it was estimated to have jumped to over 80%, improving confidence and reducing production issues. Testing more with less effort, meant fewer missed defects, reducing post-release firefighting and improving system reliability.

Conclusion

The implementation of test automation and test management tools at this healthcare provider fundamentally changed how system reliability and performance were managed. With Tricentis Tosca and a structured test management framework, testing became faster, more scalable, and highly effective at ensuring quality across Epic EHR and their 250+ integrated applications.

Test cases were reduced by 75%, eliminating lengthy manual testing delays and accelerating release validation. This resulted in over $800,000 in annual cost savings, primarily by reducing labor costs, infrastructure overhead, and error remediation. Workflow efficiency improved by 64%, allowing IT and clinical teams to collaborate more effectively and significantly streamlining the testing lifecycle.

By enhancing defect identification and resolution by 58%, the organization minimized production issues and ensured system stability with quicker response times. Test coverage expanded by 80%, validating more workflows, integrations, and compliance requirements across the hospital ecosystem.

These advancements created a measurable impact on operational efficiency and patient care. Automated testing ensured that critical healthcare systems remained stable and accurate, allowing medical professionals to focus on providing exceptional patient care without system interruptions. As regulatory demands and healthcare technology continue to evolve, this case study demonstrates how modernizing the Software Development Lifecycle is a necessary investment in the future of healthcare IT.