Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.
Handshake AI works directly with frontier AI lab researchers to create evaluations, publish benchmarks, and improve AI models through human expertise.
As an AI Model Policy Trainer, Generalist, you will turn complex customer policies into consistent, well-reasoned evaluations of AI model behavior.
You will read user requests, model responses, and relevant conversation history, then determine which policy category best applies. The most interesting cases will not have obvious answers.
What you'd do
- Learn customer policies, definitions, taxonomies, and evaluation rubrics, and adapt as projects and guidance change.
- Evaluate user requests and AI model responses within the full relevant conversation context.
- Distinguish between closely related labels, severity levels, and policy boundaries to select defensible classifications.
- Write concise, evidence-based rationales citing relevant policy language and conversation details.
- Identify policy gaps, contradictions, and emerging edge cases, and raise questions when guidance does not resolve a case.
- Participate in calibration discussions with evaluators, project leads, policy teams, and researchers; challenge interpretations respectfully and update your judgment when evidence warrants it.
- Apply customer policy consistently, maintain accuracy across repeated evaluations, and incorporate feedback into future work.
- Help improve evaluation frameworks, examples, decision rules, and quality standards.
- You enjoy precise distinctions and notice when one word, contextual detail, or change in intent affects the answer.
- You can hold a strong opinion without becoming attached to being right and discuss disagreement respectfully.