Humanities Evaluation Specialist

Responsibilities

• Develop original, high-difficulty question-and-answer pairs that challenge advanced AI models on interpretation, argument, context, and analytical reasoning within the humanities.
• Research and verify answers using primary texts, reputable secondary sources, and authoritative citations.
• Design prompts that require deep analytical or interpretive insight rather than surface-level factual recall.
• Test questions against AI models and iterate on them to increase difficulty, improve clarity, and maintain correctness.
• Craft precise and unambiguous questions together with well-supported answers.
• Clearly document reasoning and supporting sources for submitted work.
• Apply reviewer feedback to refine submissions and maintain alignment with project guidelines and quality standards.

Skills

Requirements

• Advanced academic background or professional experience in a humanities discipline such as history, literature, philosophy, linguistics, classics, art history, or journalism.
• Exceptional analytical writing, close-reading, and question-formulation skills.
• Strong research and source-triangulation capabilities.
• Ability to provide authoritative citations and clearly document reasoning and sources.
• Demonstrated attention to detail and written precision.
• Experience developing challenging and nuanced questions that test understanding beyond surface-level facts.
• Ability to work independently and reliably in a remote environment.
• Ability to meet the project's minimum weekly task submission requirements.
• Familiarity with AI model evaluation or training is beneficial but not required.
• No prior experience in AI is required.

Benefits

• Fully remote contractor opportunity.
• Compensation equivalent to $40–$90 per hour based on qualifying output.
• Opportunity to apply humanities expertise to next-generation AI training and evaluation.
• Research-intensive work involving interpretation, critical reasoning, source verification, and analytical writing.
• Opportunity to work across humanities disciplines including history, literature, philosophy, linguistics, classics, art history, and journalism.
• Selected experts may begin their first tasks within 24–48 hours after completing onboarding.
• No prior AI experience required.