Research Scientist (Remote/US/LATAM)
Anyone AiRemote
HirQ verdict
45/100 · Weak. Based on age, reposts, pay, the company's hiring and ghost signals.
- True age
- 137 days
- Reposts
- Posted once
See the evidence behind this score.
Pro shows each part of the score, the reasons, the pay percentile and the listing history.
HirQ's ghost-risk estimate and company labels are automated opinions based on what we observe on employers' public job boards: how long a listing has been open, whether it was re-listed, wording in the description, and how often roles on that board close. They are not statements about an employer's intent, and a high-risk listing can still be a real opening. Figures are as of the date shown. How we measure
At a glance
- Pay
- No pay listed
- Level
- Mid level
- Type
- Contract
- Team
- Anyone AI Internal Team
- Work setup
- Remote
“Fully Remote”
Anyone Ai hiring
- Open roles
- 227
- New in 30 days
- 24
- Closed in 30 days
- 10
About the role
From Anyone Ai's careers page, checked Nov 20
Research Scientist, LLM Evaluations & Benchmarking
Anyone AI Labs
Reports to: CEO · Remote / LatAm / US
The role Evaluation is one of the hardest open problems in AI: we still don't have reliable ways to measure what frontier models can and can't do, and the field mostly runs on benchmarks that are saturated, contaminated, or measuring the wrong thing. You'll own that problem at Anyone AI, measuring frontier model capability.
This is a research role at heart: you decide what a good evaluation is , design the benchmarks that prove it, and defend the methodology under lab scrutiny. You'll build frontier-grade evaluation packages across reasoning, coding, agents, tool use, and multi-modal — grounded in expert-verified truth, validated against multiple models, and QC'd to survive buyer-side review.
Responsibilities
- Evaluation research. Turn eval targets into original benchmark designs. Own the hard measurement questions: construct validity, item discrimination, headroom, reliability, contamination, and capability elicitation. Push toward evals that stay informative as models improve.
- Benchmark development. Build evaluation packages with subject-matter experts, each with expert-verified ground truth, multi-model headroom results, and rigorous QC (calibration layers, severity-weighted rubrics, deterministic verifiers).
- Experts. Recruit, calibrate, and review a pool across coding, agentic/tool-use, and STEM/reasoning. Be the final arbiter of correctness and frontier difficulty.
- Lab relationships. Be a technical point of contact for labs, with CEO support. Understand what they're trying to measure and translate it into an evaluation design.
- Delivery & dissemination. Turn lab requests into winning sample packages and own pilots end to end. Where the work generalizes, help turn it into public benchmarks and papers: we support publishing at venues like NeurIPS Datasets & Benchmarks, ICLR, and ACL.
What we're looking for
- Research background in ML evaluation or benchmarking (a track record of published or open benchmarks, eval/measurement research, or equivalent hands-on work that labs have relied on).
- Deep LLM/frontier-model benchmarking expertise, with real strength in code-model and agentic evaluation.
- Fluency with the measurement problem itself: construct validity, psychometrics, rubrics, pass rates, headroom, contamination, and what makes a task genuinely discriminate a model.
- Interest in the safety side of evaluation, capability elicitation, robustness, and measuring the things that are hardest to measure honestly.
- Proven ability to hold a team or expert pool to a rigorous standard.
- Comfort with the full research loop: framing the question, running the study, and writing it up.
- Fluent English; Spanish a plus.
Skills mentioned
- LLMs
- Machine Learning
- Spanish
Similar jobs
Same role or field, nearby or remote, ranked by HirQ score.
Principal Research Scientist
G2Remote (US)Remote
Research Scientist
OutsetSan FranciscoRemote
Senior Research Scientist - Risk
NearaNew York CityRemote
More at Anyone Ai
Career Mentor & Job Placement Specialist
Anyone AiArgentina - Fully RemoteRemote
Platform Engineer, RL & Eval Data | 100% remote
Anyone AiArgentina - Fully RemoteRemote
Backend Developer (Uruguay)
Anyone AiUruguayRemote
Backend Developer (Peru)
Anyone AiPeru - Fully RemoteRemote
Keep exploring
More jobs at Anyone Ai
- Career Mentor & Job Placement SpecialistRemote
- Platform Engineer, RL & Eval Data | 100% remoteRemote
- Backend Developer (Mexico)Remote
- Backend Developer (Colombia)Remote
- Backend Developer (Brazil)Remote
- Backend Developer (Argentina)Remote
- Admissions Advisor – AI Fellowship (100% remote)Remote
- Senior Collections and Debt Recovery SpecialistRemote
Explore similar jobs
open jobsWhat this role pays
pay pointsCompany
open jobsResearch Scientist (Remote/US/LATAM)
No pay listed · HirQ 45