Quick overview
Job description
Posted: 01/09/2026
Closing Date: 05/12/2026
Job Type: Permanent - Full Time
Location: Sydney
Job Category: IT
AutomicGroup is Australia's market leader in technology-driven investor administration. Combining proprietary cloud-native technology with expert service,Automicsimplifies investor administration and governance for companies and funds. We provide the market's leading registry, employee share plans, IR analytics, and fund administration services for over 1,400 ASX-listed and unlisted organisations.
Automic'sculture is collaborative, inclusive, and purpose driven. It is grounded in our mission to connect companies and funds to their investors through technology, service, and trust, and guided by our values ofOneAutomic, Go Beyond, Empowered, and Accountable.
Job DescriptionAbout the Role
We'relooking for aSenior AI Test Engineer to provide independent assurance thatAutomic'sAI-enabled products, agents and platforms are reliable,secureand production-ready.
Working closely with our AI Engineering team,you'llshape quality strategy,establishrobust AI evaluation and testing practices, and provide evidence-based release and residual-risk advice.You'llplay a key role in ensuring our AI solutions deliver safe,reliableand high-quality outcomes.
This role requires direct experience testing AI systems. Using AI coding assistants or generative AI to write test cases, scripts or documentation alone does not meet this requirement.
Key Responsibilitiesinclude:
Lead risk-based quality strategies for AI applications, agents, retrievalsystemsand supporting services.
Deliver functional and non-functional assurance across integrations and end-to-end workflows.
Establish AI evaluation datasets, test data, qualitythresholdsand automated regression coverage.
Assess AI response quality and safety, including hallucination,groundedness, adversarial behaviour and sensitive-data exposure.
Validate agent workflows, authentication, authorisation, humanapprovalsand tenant isolation.
Drive automated quality controls, defectmanagementand continuous improvement across delivery pipelines.
Provide evidence-based release recommendations and investigate production quality issues.
Desired Skills and ExperienceAbout you!
To be successful in this role,you will have:
Hands-on experience designing and running tests or evaluations for generative AI, LLM, agentic or RAG applications, with clear examples of your personal contribution. This is essential; using AI tools to assist conventional software testing alone is not sufficient.
Proven experience in quality engineering or test automation for complex production systems.
Practical experience evaluating AI response accuracy, relevance, groundedness, hallucinations and safety.
Experience building evaluation datasets, defining quality thresholds and automating regression tests that account for variation in AI outputs.
Programming experience in Python or another language used for test automation and service testing.
Experience testing APIs, integrations and end-to-end workflows, including security, resilience and performance.
Strong analytical and communication skills, with the ability to explain findings, work through defects with engineers and make evidence-based release recommendations.
A tertiary qualification in technology, engineering or a related discipline, or equivalent professional experience.
Experience in financial services or another regulated environment is desirable.
You'llbe part of a team whereoperational excellence, collaboration, and continuous improvementare genuinely valued. This is an opportunity to deepen yourexpertise, contribute to meaningful outcomes, and grow your career within a high-performing, purpose-led fintech environment.
Key Benefits include:
Professional career growth & development
Supportive & fast paced environment
Opportunity to grow with the business
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- DR
Staff Software EngineerRecently posted
dreamerlabs · Sydney · On-site · Posted 21 Oct 2026 🇦🇺
Next.jsAWSPython+3TechnologySalary not listed - GT

Director, Cybersecurity Advisory &AI EnabledDeliveryRecently posted
Grant Thornton Australia Limited · Sydney · Hybrid 🇦🇺
GDPRLLMPenetration Testing+2TechnologySalary not listed - DC

CONTRACT - Senior Functional Consultant - Dynamics 365 FinOpsRecently posted
Datacom Connect · Brisbane · Remote · Posted 19 Nov 2026 🇦🇺
Stakeholder ManagementProcurementSupply Chain ManagementTechnologySalary not listed - AM
Software Development Engineer Ii, Ads Ai Core InfraRecently posted
Amazon Inc. · Melbourne · Hybrid · Posted 18 Nov 2026 🇦🇺
AWSAgileTechnologySalary not listed - EC

Senior Network EngineerRecently posted
e2 Cyber · Canberra, Australian Capital Territory · Hybrid 🇦🇺
MFAAnsibleBash+3TechnologySalary not listed - AT

Senior Solutions ArchitectRecently posted
Atlassian · Sydney · Hybrid · Posted 7 Nov 2026 🇦🇺
RubySQLGit+4TechnologySalary not listed - MI

Senior Backend EngineerRecently posted Auto Apply
Midnite · Australia · Remote · Posted 18 Nov 2026 🇦🇺
DockerFastAPIFlask+10TechnologyA$180k - A$200k/yr - CO

Business Systems AnalystRecently posted Auto Apply
City of Stirling · Stirling, Western Australia · On-site · Posted 19 Nov 2026 🇦🇺
Stakeholder ManagementSQLBusiness Analysis+6TechnologyA$115.5k/yr - T&
Senior Cost Manager - MEP - Data CentreRecently posted Auto Apply
Turner & Townsend · Melbourne, Australia & New Zealand · On-site · Posted 19 Nov 2026 🇦🇺
ProcurementTechnologySalary not listed - AE
Technology & Security ConsultantRecently posted Auto Apply
AECOM · Fortitude Valley, QLD · On-site · Posted 18 Nov 2026 🇦🇺
TechnologySalary not listed - ER
Technical Authority Ran Sydney,New South Wales,Australia Service Delivery Posted 17 mins agoRecently posted
Ericsson · Sydney · Hybrid · Posted 18 Nov 2026 🇦🇺
Stakeholder Management5GLoad BalancingTechnologySalary not listed - SU
Infrastructure and Cloud, ICT Cyber Security and Systems (Various Roles)Recently posted
Style Up · Canberra, Australian Capital Territory · Hybrid · Posted 18 Nov 2026 🇦🇺
SharePointTechnologySalary not listed
