Technical Program Manager, Frontier Evals

This role is based in San Francisco, CA, requiring 3 days in the office per week.

Compensation:

$207K - $230K/year

Responsibilities:

• Manage frontier evaluation projects from initial research questions to delivered benchmarks.
• Partner with researchers and engineers to translate ambiguous model capability questions into concrete eval designs, success metrics, timelines, and execution plans.
• Design and manage human data campaigns, including task design and quality control workflows.
• Do hands-on technical work including data analysis and debugging eval pipelines.
• Coordinate across various teams to deliver high-quality evals under tight timelines.

Requirements:

• Experience in technical program management or similar roles.
• Proficiency in Python, SQL, or similar tools for data analysis.
• Strong understanding of large language models and evaluation methods.
• Ability to communicate effectively with technical and non-technical stakeholders.