Principal Software Engineer for Copilot Evals

CADET (Customer and Analytics Driven Evals Team) is building a customer-grounded quality system for Copilot. Our mission is to rapidly identify the customer scenarios that matter most, represent them faithfully in evaluation and learning assets, run quality gates continuously, and turn every important failure into reusable product and model improvements. We bring together DSAT and other product signals, deep customer engagements to create representative eval sets. Operating in a fast-paced environment, we connect customer grounded quality issues with quality teams to advance Copilot quality and product innovation.

We are looking for a Principal Software Engineer to architect and build the production system behind CADET's (Customer and Analytics Driven Evals Team) customer-grounded quality loop, partnering with our platform teams. This is a high impact role that will set technical direction, drive rapid delivery, and align engineering, science, product, and platform teams as we shift from manually assembled signals and one-time evaluations to an agentic, automated system. You will connect heterogeneous data sources, AI agents, evaluation infrastructure, RLE platforms, model and product telemetry, human review, and quality reporting into a reliable and inspectable end-to-end system. You will set architecture across organizational boundaries, establish engineering standards, align partner teams, and drive the most technically demanding parts of the system.