Evaluation Specialist/Recent Grad

Responsibilities

• Develop original and challenging question-and-answer pairs that test advanced AI models across diverse subject areas.
• Conduct in-depth research and triangulate multiple sources to ensure answers are accurate, comprehensive, and well documented.
• Create multi-step questions requiring synthesis and analytical reasoning rather than simple single-source lookups.
• Test questions against AI models, analyze their responses, and iterate to adjust difficulty and effectiveness.
• Document research processes with clear citations and logical explanations supporting each answer.
• Refine submitted content based on reviewer feedback while maintaining project guidelines and quality standards.

Skills

Requirements

• Demonstrated ability to conduct thorough independent research and critically evaluate information from multiple sources.
• Exceptional attention to detail and written precision with fluency in English.
• Ability to create nuanced, well-structured questions that test deep understanding and analytical reasoning.
• Strong self-direction, reliability, and accountability when working independently in a remote environment.
• Interest in advanced AI systems and evaluating the boundaries of current model capabilities.
• Open to recent graduates, advanced-degree holders, and candidates from any academic or professional background with strong research and writing skills.
• Prior experience in AI training, evaluation, or content creation is advantageous but not required.

Benefits

• Fully remote contractor opportunity.
• Open to recent graduates and candidates from diverse academic or professional backgrounds.
• Opportunity to work directly on advanced AI evaluation and research tasks.
• Fast onboarding, with selected contributors generally expected to begin within 24–48 hours after onboarding.
• No prior AI training experience required.