Sportsbet accelerates production releases with Quality Engineering strategy and automated testing

Agile innovation is essential to Sportsbet’s competitiveness in the lightning-paced hyper-competitive online betting market. Sportsbet maintains its market leadership position by attracting and engaging customers with regular product and feature innovations.

Sportsbet’s Core Betting Team is responsible for developing the software applications and services that enable these new betting features in the mobile apps and website. The team has built a reputation for rapid development of applications and services, thanks to an Agile project delivery model, highly flexible platform architectures, and high levels of code reuse across platforms.

Hampered pace of innovation

However, the team grew concerned that release schedules were under pressure due to timing challenges for quality assurance and testing. The pace of innovation was being hampered by testing practices that weren’t in step with the high expectations for their Agile DevOps.

The Delivery Lead for the Core Betting Team identified some areas for improvement in their approach to QA and testing:

• While all developers and testers in the team were involved in manual testing, it was being executed as needed, with scope to improve planning, structure, record keeping and management.
• Unit testing was automated. However, API testing had a complex framework with a steep learning curve.
• An available Xray test management tool was not being utilised due to unawareness of the utility of the tool.
• Testing was not a formalised part of sprint commitments.
• The Testing Strategy required development.

To improve their efficiency and effectiveness with QA and testing, Sportsbet decided they needed to implement an overall Quality Engineering (QE) strategy, introduce more automation, and improve their DevOps practices.

To ensure they were implementing best practices, Sportsbet approached Shine for assistance with all three. Having had Shine’s front-end and back-end engineers augmenting their development teams for nine years, Sportsbet was accustomed to elevating its DevOps capability with the best practice expertise Shine brings from experience with diverse enterprise clients.

Developing a Quality Engineering strategy

In close consultation with the Core Betting Team’s delivery lead, Shine provided a senior quality engineer to lead the QE strategy development and design manual and automated testing practices.

The first key pillar of the QE strategy was to establish a ‘one-team approach’ where quality becomes the whole team’s responsibility. This was complemented by implementing a Shift-left testing approach to find and prevent defects earlier in the software delivery process. Both were achieved by having ‘the three amigos’ – BAs, QAs and Developers – involved in all aspects of testing and ensuring testing happens as early as possible.

Shine’s senior quality engineer tailored the overall QE approach by:

• Writing a guide for how to convert Agile user story requirements into test cases.
• Defining the process for creating and running tests within sprints.
• Creating a methodology for identifying, discussing and prioritising defects within a sprint.
• Identifying the approach for writing and storing test artefacts – test cases, test plans etc – within XRay.
• Defining an automation-first approach for regression scenarios.
• Identifying integration points where end-to-end testing can take place.
• Providing t-shirt size estimates for the project epics.

Establishing a new automation framework

In applying a DevOps approach, Sportsbet’s Core Betting Team already had an automation framework for one API service built using Karate. However, when the individual who initially set up the framework left the organisation, a knowledge gap emerged, and the Karate-based automation suite was not maintained. This resulted in Jenkins builds failing continuously because Karate automated tests were failing when called.

As the API services were developed using Java and the Core Betting Team comprised Java developers, Shine’s senior quality engineer guided the team’s engineers in developing a Java-based API testing automation framework using JUnit in combination with OkHttp.

The work involved creating builder and model classes for the APIs and a utilities package to contain the OkHttp common methods with different API request builders. This framework required significant AWS integration, with AWS Systems Manager used for storing secrets and credentials and SAML used for authentication. It also involves CI/CD implementation using Jenkins for virtualisation, where the tests run each time a new build is detected on the API service. Allure Framework test reports are generated in Jenkins, and the Xray test management tool is now being used to import and log JUnit test results.

Embedded in the development team

Once these QE practices and frameworks were established, Shine’s senior quality engineer was embedded in the development team to contribute the following during each sprint:

• Tweak the above QE approaches as needed.
• Guide the developers in the estimation of story points and in choosing an appropriate sprint commitment.
• Perform hands-on testing to help guide and assist the developers in creating and carrying out test cases.
• Help prioritise bugs with the product owner.
• Work with the business analyst to review tests against acceptance criteria.
• Identify tests for regression and automation.
• Consult with other application development teams to organise end-to-end testing.

One-team approach to quality

After three months of working with Sportsbet’s Core Betting Team, Shine’s senior quality engineer guided them to successfully establish a ‘one team’ approach where quality is the whole team’s responsibility. Test automation is now established as a sprint commitment, and developers are actively engaged in unit testing, functional testing and automating regression tests. Testers are involved in code review, functional testing and functional test automation.

With this shift-left approach in place, the Core Betting Team is achieving 100% automation of repeatable critical regression tests within project sprints. All this translates to production release schedules being completed on target as the Core Betting Team accelerates the pace of innovation at Sportsbet.