About the role
Project Overview:
Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you’ll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy. We are seeking a skilled Licensed US-Based Attorney to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position — you will be engaged as an expert project consultant on a contract basis.
Location:
U.S.-based experts only Engagement: Part-time, 1099 project-based expert evaluation work
Work Type: Remote
Project Summary:
This project supports the development of an AI benchmarking framework designed to measure how effectively artificial intelligence models perform real-world, economically valuable knowledge work. The benchmark evaluates AI systems against tasks commonly performed by legal professionals in business environments. Contributors will create expert benchmark outputs, define evaluation criteria, and assess AI-generated work products across a range of professional scenarios. The goal is to establish rigorous, expert-driven standards for measuring AI performance on complex business and regulatory workflows. Consultant Engagement Terms This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).
Responsibilities:
Contributors will: Create benchmark answers for realistic legal scenarios Develop scoring rubrics defining high-quality legal work product Evaluate AI-generated legal analyses and written deliverables Identify legal inaccuracies, unsupported conclusions, and material omissions Assess legal reasoning, risk identification, and practical applicability Provide written justifications for scores and evaluation decisions Participate in calibration and quality review exercises Expected Outcomes High-quality benchmark answers reflecting professional legal practices Clear, defensible evaluation rubrics aligned with real-world business requirements Consistent and reliable assessment of AI-generated work products Identification of strengths, weaknesses, and risk areas in model performance Expert evaluations that support rigorous benchmarking of AI systems on economically valuable knowledge work Documentation that enables repeatable and scalable model evaluation
Qualifications:
Active U.S. law license in good standing Minimum 3+ years of professional legal experience Strong legal research, analysis, and written communication skills Ability to assess legal arguments for accuracy, completeness, and sound reasoning Experience reviewing contracts, policies, memoranda, legal correspondence, regulatory guidance, or legal analyses Strong attention to detail and professional judgmentU.S.-based
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.
More like this