Data Scientist, Evaluations - Meta Superintelligence Labs

This role is based in Menlo Park, CA.

Compensation:

$177,000/year to $247,000/year + bonus + equity + benefits

Responsibilities:

• Lead the design of evaluation stimuli and benchmarks with minimal bias.
• Design and execute sampling strategies to measure model performance.
• Perform deep-dive analysis on model errors and quality gaps.
• Collaborate with Research Scientists and Engineers to create scientifically sound benchmarks.
• Drive the publication of novel evaluation research.

Minimum Qualifications:

• Bachelor's degree in Computer Science, Mathematics, or a relevant technical field.
• 6+ years of experience in analytics (4+ years with a Ph.D.).
• Experience with SQL, Python, and statistical software.

Preferred Qualifications:

• Master’s or Ph.D. in a quantitative field.
• Publication record in top-tier peer-reviewed venues.
• Expertise in language model evaluation or psychometrics.