Senior Research Engineer - Evaluations

This role is based in San Francisco, CA with a hybrid work model.

Compensation:

$220,000 - $280,000/year

About the role:

Engineer AI agents to assess the quality and alignment of generative design models.

Responsibilities:

• Design and optimize an 'MLLM-as-a-Judge' evaluation system.
• Implement inference-time alignment techniques to improve model output quality.
• Establish a benchmarking process for foundation models.
• Analyze evaluation data and provide actionable insights.
• Collaborate with research scientists and ML engineers.

Requirements:

• Strong understanding of generative AI models and architectures.
• Experience with PyTorch and large-scale distributed model training.
• Familiarity with cloud environments, ideally AWS.