Keysight will be at NRF 2024, New York, showcasing our automated POS testing solution. Visit level 1, booth 632 to see a live demo and an opportunity to discuss our latest report’s findings.
Point-of-sale (POS) systems have expanded beyond the simple cash register. Now, they are the central hubs, connecting to essential in-store peripheral devices and back-end systems while bridging the gap between online purchases and in-store pick-ups.
As a result, retailers must perform rigorous end-to-end testing to achieve operational excellence. However, despite advances in AI-augmented software testing and the financial implications of slow releases crawling with bugs, 72% of North American retailers persist with manual testing.
Keysight commissioned NRF SmartBrief to survey software testing decision makers at North American retailers to research how their current POS testing approach impacts their capabilities of delivering high-quality software quickly. The resulting study, “Human versus Machine: The State of POS Testing,” and its findings are summarized below.
An Impossible Decision
Software testing is an essential part of the software development lifecycle for any POS system. It ensures that releases meet end-user requirements, are of high quality, and are free from critical defects.
Nevertheless, when faced with an impossible decision of faster releases or fewer defects, 81% of senior decision-makers chose speed over quality. Considering that most North American retailers perform manual testing, the study shows compelling evidence that these techniques contradict any chance of software delivery being fast enough or of sufficient quality.
Percentage of responses indicating their preferred benefits from automation
Manual Testers Fight a Losing Battle
Persisting with manual testing when you want faster releases is one of life’s mysteries. Of course, there will be painless, quick, and error-free releases. However, the study shows that, on average, retailers invest 34 hours of testing per release. Testing for nearly five working days, you’d think all test requirements would be met before a release. Sadly, this isn’t the case. Manual testing creates bottlenecks that delay the deployment of new features, POS functionality improvements, and critical bug fixes.
The main culprit of slow and insufficient manual testing is the considerable proportion dedicated to the crucial but often labor-intensive task of regression testing. Seventy-one percent of testing per release is solely devoted to regressions, reducing testing time for new POS features and system enhancements. As senior decision-makers demand release velocity increases, the pressure on QA teams intensifies. The lack of time for testing new features hampers the ability to keep pace with required releases, resulting in 69% of manual testing teams only managing to release five times or fewer each year.
The number of releases manual testing teams can acheive in a year
When using Google Cloud and DORA metrics for release frequency to benchmark against other organizations, most retailers cannot achieve elite or high DevOps performance while continuing with manual testing methods. In an environment where test requirements frequently change, as do customer demands, a retailer’s inability to react quickly to changing market forces should be of great concern.
When Manual Testing Falls Short, Automation Delivers
While retail QA leaders emphasize faster releases, the benefits of automation extend beyond this goal. Test automation offers speed and quality, enhancing test coverage across the POS ecosystem.
Automation effectively addresses challenges in manually testing new features and identifying critical defects. An impressive 81% of respondents using automation express high satisfaction with the achieved test coverage. Automation facilitates a more robust and reliable POS system by swiftly identifying bugs and aiding the deployment of new functionality.
Percentage of respondents using automated testing who are satisfied with their organizations capabilities in specific areas
QA teams leveraging automation also boast a strong satisfaction rating of 79% in their ability to scale testing efforts. Contrastingly, manual testing teams struggle to adapt to the growing complexity of POS systems, hampered by detailing every step in complex manual scripts. Changing test requirements complicates the issue further, causing release delays or rollbacks.
Meeting test requirements before a release consistently challenges manual testers, impacting deployment speed and frequency. In contrast, 73% of teams using automation are highly satisfied with their ability to meet test requirements, contributing to POS system stability and minimal disruptions at the checkout.
The Future Dilemma: Surges in Release Cycles Spell Disaster
As retailers face the pressure to increase release cadence in the coming years, the shortcomings of manual testing become glaringly obvious. Delays and cancellations plaguing manual testing teams pose a formidable challenge, especially with 72% of senior decision-makers expecting annual releases to reach 16 or more within three years.
Assessing teams’ readiness to handle more releases reveals a stark disparity in testing effectiveness. Fifty-one percent of teams using automation have the capabilities to reach the target of 16 or more, compared to a mere 23% relying on manual techniques.
The lack of capabilities for manual testers is exacerbated by the time required to reach this target. Currently, 47% of manual testing teams will need at least an additional 32 weeks of testing, making it impossible to achieve the proposed annual release cycle within three years.
Current manual testing time will prevent retailers from reaching the goal of 16 releases or more
This study delivers a significant revelation that should concern 72% of retailers. Without transitioning from manual testing to automation, they will lack the capability to release enough to satisfy their customers in the future.
Download the Human versus Machine: The State of POS Testing study today and put this research into practice to optimize your testing teams and the services you provide to your customers.