The CIO Division delivers a broad range of enabling services and operational support to the department and Australian businesses. The division is currently seeking an experienced and motivated Automation Tester to contribute to the delivery of various ICT projects. The NMI Web Services Enhancement Program has a mixed skill development team of 14 that reimagines how the National Measurement Institute delivers regulatory, calibration, and trade measurement services across Australia. By integrating a secure, user-centric web portal with Microsoft Dynamics 365 CRM and digitising key workflows, the project empowers businesses and regulators with real-time access to services, data, and insights. It replaces fragmented, manual processes with streamlined, automated solutions enhancing transparency, accelerating compliance, and strengthening Australia's measurement infrastructure to support innovation, global trade, and public trust. The Senior DevOps Automation Test Engineer will play a key role in advancing both functional and non-functional automated testing capabilities for the NMI Web Services Enhancement Project. The successful candidate will have demonstrated experience and responsibilities in designing and configuring test automation scripts for React web applications, mobile applications, web services, Microsoft Dynamics 365, Power Automate, and Canvas Apps, as well as conducting end-to-end testing of complex D365 CRM implementations and CI/CD integration of test automation suites, and performance benchmarking. Requirements Key duties and responsibilities We are looking for an Automation Tester who can: Automation Testing (Functional): Construct a complex suite of automation tests using TypeScript, C#, Python, ASP.NET 8 Core, YAML for cloud-based, client-facing React web portal built integrated with Microsoft Dynamics 365 CRM to confirm that the end-to-end process, application, systems, and services function appropriately and validate the environment in which systems and services operate. Design, develop, and automate testing using browser-automation frameworks such as Microsoft Playwright and Selenium WebDriver. Design, develop, and automate user interface tests for Microsoft Dynamics 365 CRM and other model-driven Power Platform applications using the Easy Repro framework Integrate and manage the automation testing suites and framework CI/CD pipelines for nightly regression functional testing using Azure DevOps. Monitor and report test results, log defects for each release, and collaborate with manual testers and developers to investigate issues by replicating them and reviewing the code to identify the root cause. Lead automation initiatives to enhance testing strategy by identifying opportunities with stakeholders and implementing best practices in automation testing. Non-Functional Testing: Design and execute performance, load, and stress tests using JMeter or Azure Load testing. Analyse performance results and provide actionable insights. Configure and utilize Azure Application Insights to Monitor for performance metrics and diagnostics. Design and implement automated test scripts for validating end-to-end API integration points between, systems, using tools such as Postman. Promote web accessibility and test for conformance to web accessibility guidelines using tools like WAVE, NVDA, SortSite Set up Azure DevOps CI/CD pipelines for non-functional testing, test management, and automation across all applications. Where required validate security remediations. Other Attributes: Proactive self-starter who excels in fast-paced environments, effectively manages multiple priorities, and is committed to delivering high-quality software through continuous improvement and technical excellence. Display curiosity, creativity, persistence, open-mindedness, analytical ability, unconventional thinking, clear communication, logic, and attention to detail. They can synthesize information and make various deductions, as well as understand the potential outcomes resulting from specific inputs. Work closely with the Business and Delivery team to understand requirements and translate them into appropriate automated tests including creation of test data. Use Azure DevOps daily for managing user stories, test plans, source control, and release pipelines as part of a multidisciplinary team. Champion a culture of continuous improvement by identifying process inefficiencies, implementing refinements to testing practices, and leveraging feedback to enhance quality and delivery over time. The approximate breakdown is as follows Functional Testing (75%): Automation of UI and End-to-End Tests: The core of the role will involve automating functional testing tasks. This includes: • Uplift, develop, and automate test suites for a client-facing React web portal integrated with Microsoft Dynamics 365 CRM. o Tools: Microsoft Playwright, Selenium WebDriver o Languages: TypeScript, C#, Python, ASP.NET 8 Core • Automate user interface tests for Microsoft Dynamics 365 CRM and model-driven Power Platform applications using the Microsoft EasyRepro framework. • Log defects, report test results, and collaborate with developers to investigate and resolve issues through code review and replication. • Configure and maintain Azure DevOps CI/CD pipelines using YAML to embed automated functional regression testing into nightly execution cycles including the orchestration of test suites is crucial to the role. • Test data creation and management to support reusable automation scenarios across environments. Non-Functional Testing (20%): These tasks, while critical, are more targeted. They encompass: • Execute existing JMeter test suites and lead the migration to Azure Load Testing, integrating with Azure Application Insights for real-time telemetry, diagnostics, and performance analytics. • Set up and maintain Azure DevOps CI/CD pipelines to support non-functional testing. • Conduct accessibility testing to meet WCAG 2.2 AA compliance using established toolsets including WAVE, NVDA, and SortSite. • Design and execute automated end-to-end API integration tests using Postman. Where required, validate security remediations in accordance with Australian Government Information Security Manual (ISM) controls. Test Strategy Development (5%): Strategic Leadership in Automation • Lead initiatives to uplift the testing strategy by identifying opportunities for improvement in automation testing practices, frameworks, and tooling to enhance test coverage, reliability, and maintainability. Key deliverables Build and configure automated test plans in Azure DevOps. Refactor and extend Playwright and Selenium test suites to support nightly automated regression runs. Develop and automate user interface tests for Dynamics 365 CRM and other model-driven Power Platform applications using the EasyRepro framework Integrate Playwright, Selenium, Lighthouse, and EasyRepro tests into Azure DevOps pipelines for nightly regression testing with robust reporting and failure diagnostics. Implement robust test data management strategies to support automation testing tools and ensure consistent, reusable test scenarios. Refactor and extend current performance tests to include stress testing and scalability scenarios to validate system performance under peak load conditions. Lead the migration from JMeter to Azure Load Testing including configure Azure Load Testing to integrate with App Insights for real-time telemetry, diagnostics, and performance analytics. Conduct cross-platform UI testing across mobile and desktop environments, including comprehensive browser compatibility validation. Web accessibility and testing to meet Web Content Accessibility Guidelines (WCAG) 2.2 AA using tools like WAVE, NVDA, SortSite (Accessibility and inclusion SFIA Level 3) Implement Azure DevOps management and strategies to detect, isolate, and report flaky tests. Maintenance and uplift of the test tool suite. Coordination and reporting of release activities across delivery tranches. Criteria The buyer has specified that each candidate must provide a one page pitch to address all criteria specified. This is equal to 5000 characters. Essential criteria 1. Demonstrated experience leading functional and end-to-end testing activities across a range of platforms and technologies including user interfaces (e.g. web applications, mobile applications, web services and Microsoft Dynamics 365, Power Automate, and Canvas Apps.) How did you develop, automate, and execute end-to-end test plans ensuring alignment with business and system requirements. (Process and Functional testing SFIA Level 5) 2. Demonstrated experience in developing and maintaining automated regression tests and scripts for web applications, using browser-automation frameworks such as Selenium and Playwright. How did your scripting contribute to automation efficiency or test coverage? What tools, languages and frameworks did you use (Programming/Software Development – SFIA Level 4) 3. Demonstrated experience in developing and maintaining automated test for Dynamics 365 using Easy Repro. How did your scripting contribute to automation efficiency or test coverage? (Methods and Tools SFIA Level 4) 4. Demonstrate experience integrating automated test suites within Azure DevOps pipelines for regression and release testing, what role did you play in integrating into the CI/CD pipelines? What challenges did you face in test orchestration, environment setup, or tool integration (Systems Integration Build – SFIA Level 4) 5. Demonstrate experience in planning and executing non-functional distributed performance and load testing using tools such as JMeter or Azure Load Testing. How did your testing contribute to system quality, compliance, and performance benchmarks? (Non-Functional Testing SFIA Level 5) Desirable criteria 1. Experience in developing tests cases using Cucumber or writing Acceptance Criteria in Gherkin format. (Methods and Tools SFIA Level 4) 2. Demonstrate experience and ability to work to a release cadence, to set priorities and to meet delivery timelines both independently and as a member of a multi-disciplinary team. (Release Management - SFIA Level 3) 3. Demonstrated knowledge of web accessibility and testing to meet Web Content Accessibility Guidelines (WCAG) 2.2 AA using tools like WAVE, NVDA, SortSite. (Accessibility and inclusion SFIA Level 3)
