AI Quality & Reliability Engineer (QA/SRE)

We are seeking an AI Quality & Reliability Engineer to own the "Production Readiness" of our cloud-based AI solutions. This hybrid role combines automated software testing, AI model evaluation, and Site Reliability Engineering (SRE). You will build the automated frameworks that validate our AI outputs and ensure the underlying Azure/AWS infrastructure is resilient, performant, and compliant with banking standards.

Key Responsibilities

• AI Quality Automation: Design and execute automated testing frameworks for AI services (e.g., Azure OpenAI, AWS Bedrock). This includes testing for model hallucinations, accuracy, and "AI Content Safety" latency.
• Resiliency Engineering (SRE): Implement "Chaos Engineering" and load testing to ensure web/mobile backends can handle banking-scale traffic. Maintain high availability through automated recovery scripts.
• Automated Regression: Build CI/CD-integrated test suites using Python that validate both the application logic and the infrastructure state (IaC validation).
• Observability & SLIs: Define and monitor Service Level Indicators (SLIs) and Objectives (SLOs). Set up advanced alerting in Azure Monitor or AWS CloudWatch to catch performance degradation before users do.
• Security & Compliance Testing: Automate security scans and compliance checks to ensure all AI data handling meets strict banking data residency and privacy protocols.