P
Listed on Protege’s own careers site. You apply with them directly — we never stand between you and the employer.
What this role is
This role involves designing datasets, environments, and evaluation frameworks to assess how well AI agents perform on realistic tasks. It suits researchers with machine learning expertise and hands-on experience in reinforcement learning or agentic systems who want to tackle the foundational problem of data quality for advanced AI.
Our summary, not Protege’s wording. The full posting is on their site.
Skills this role names
Log in to see which of these are already on your profile.
What they ask for
Required
- PhD in machine learning, computer science, statistics, engineering, mathematics, economics or related quantitative field, or Master's degree with 4+ years industry experience
- Understanding of AI model training pipelines and evaluation methodology
- Experience with large unstructured or semi-structured datasets for ML training or evaluation
- Experience with reinforcement learning, sequential decision-making, or agentic systems
- Experience designing benchmarks, environments, or evaluation frameworks
- Strong experimental design and data-validation skills
- High ownership and ability to independently identify and solve high-impact problems
Nice to have
- Experience developing evaluation frameworks or performance metrics for agentic systems
- Experience translating real-world workflows into structured tasks or environments
- Experience with RLHF, RLAIF, imitation learning, reward modeling, or offline RL
- Experience with Harbor or other agent evaluation frameworks
- Publications or open-source contributions in reinforcement learning, agents, evaluation, or data-centric AI
- Experience collaborating cross-functionally with product, infrastructure, or partnership teams
- Experience with synthetic data generation, trajectory generation, or simulation-based environments