AI Research and Evaluation Analyst

Title: AI Research & Evaluation Analyst — Remote, Project-Based

Location: United States 

Department: Production

Job Description:

We are looking for highly analytical professionals to support research and evaluation work for advanced AI and large language model (LLM) projects.

This work requires more than traditional data annotation. Contributors must be able to understand complex information, evaluate factual claims and AI-generated content, apply sound judgment, and clearly explain their decisions in English.

Depending on project needs and demonstrated skills, contributors may support research and rubric development or rubric-based content evaluation.

Previous AI or data annotation experience is helpful but not required. Strong research ability, analytical reasoning, attention to detail, and exceptional English comprehension and writing skills are more important.

What You'll Do

Assignments may include two closely related types of work.

Research & Rubric Development

In research-oriented assignments, you may:

• Research unfamiliar topics using credible and authoritative sources.

• Verify factual claims and cross-reference information across multiple sources.

• Distinguish reliable evidence from weak, unsupported, or misleading information.

• Analyze complex content and identify the criteria necessary to evaluate its quality or accuracy.

• Develop clear, objective evaluation criteria and rubrics based on research and project guidelines.

• Identify factual, reasoning, sourcing, or completeness issues in AI-generated content.

• Write clear evaluation standards that can be consistently applied by other contributors.

• Document evidence, sources, and reasoning supporting your conclusions.

This work requires particularly strong research, synthesis, analytical writing, and authoring skills.

Content & LLM Evaluation

In evaluation-oriented assignments, you may:

• Review reports, responses, or other AI-generated content against predefined rubrics.

• Evaluate factual accuracy, completeness, reasoning, relevance, and adherence to instructions.

• Apply evaluation criteria consistently across multiple tasks.

• Identify unsupported claims, inconsistencies, errors, or omissions.

• Verify information using credible sources when required.

• Assign appropriate evaluations based on established guidelines.

• Provide concise, evidence-based rationales explaining your decisions.

• Maintain consistency and attention to detail across high volumes of evaluation work.

This work places greater emphasis on reading comprehension, judgment, consistency, attention to detail, and concise written reasoning.

Who We're Looking For

Strong candidates may come from backgrounds such as:

• Research or research analysis

• Fact-checking and source verification

• Investigative or editorial research

• Journalism

• OSINT or investigative intelligence

• Due diligence or background research

• Competitive or market intelligence

• Academic or policy research

• Legal research

• Information science or library research

• Trust & Safety or content quality

• AI/LLM evaluation

• Complex data annotation or quality assurance

Candidates from other professional backgrounds are encouraged to apply if they can demonstrate the required research, reasoning, and written communication skills.

Required Qualifications

• Fluent to near-native professional English proficiency (C1/C2 level or equivalent).

• Exceptional English reading comprehension, including the ability to understand nuance, qualifiers, ambiguity, and complex written claims.

• Strong analytical and critical-thinking skills.

• Ability to distinguish factual evidence from assumptions, opinions, and unsupported claims.

• Ability to produce clear, precise, grammatically strong written explanations.

• Strong attention to detail.

• Ability to follow complex written guidelines and apply them consistently.

• Ability to work independently and make evidence-based judgments.

• Strong digital literacy and online research skills.

• Ability to learn unfamiliar subject matter quickly.

English proficiency will be evaluated through a practical assessment; formal language certification is not required.

Additional Skills for Research & Rubric Development Assignments

Candidates considered for research and rubric-authoring work should also demonstrate:

• Advanced web research and source-verification skills.

• Ability to identify authoritative sources appropriate to a specific claim.

• Ability to synthesize information from multiple or conflicting sources.

• Ability to translate complex research findings into clear evaluation criteria.

• Strong structured writing and authoring skills.

• Ability to anticipate ambiguity and create criteria that can be consistently interpreted by other evaluators.

Preferred Experience

Experience in one or more of the following is beneficial:

• Fact-checking or professional source verification

• Research analysis

• Evaluation or quality review

• Rubric or guideline development

• AI/LLM evaluation or training

• Investigative research

• Content quality or Trust & Safety

• Academic, legal, policy, or market research

• Working with primary sources, government databases, academic publications, corporate records, or other authoritative sources

Previous AI experience is not required if you can demonstrate strong research, analytical, and evaluation capabilities.

Assessment Process

Qualified applicants may be invited to complete a practical online assessment.

The assessment may evaluate:

• English reading comprehension and written communication

• Analytical reasoning

• Fact-checking and source verification

• Source-quality judgment

• Evaluation of AI-generated content

• Ability to follow and consistently apply detailed criteria

• Ability to provide clear and concise rationales

Candidates being considered for research and rubric-development assignments may complete additional exercises assessing research depth, synthesis, and rubric authoring.

What Success Looks Like

Successful contributors don't simply identify whether something appears right or wrong. They can understand why, identify the evidence supporting that conclusion, and communicate their reasoning clearly.

For research-oriented work, this means being able to research a topic and help define what a high-quality evaluation should measure.

For evaluation-oriented work, this means being able to apply established criteria accurately, consistently, and efficiently while providing a concise rationale for the decision.

Engagement

This is a remote, project-based opportunity supporting AI research and evaluation projects.

Project availability, workload, duration, and schedules may vary based on client requirements. Successful qualification does not guarantee a specific volume or duration of work.

Project-specific requirements, rates, schedules, and guidelines will be communicated before assignment.