Skip to main content

About the role

Job Description: Job Description
Job Title: AI Agent Tester – Real-World Tasks Role Type: Contractor
Location: United StatesJob Summary We are seeking detail-oriented, U.S.-based AI Agent Testers to evaluate how AI assistants handle everyday tasks. You will test two AI agents using your own accounts and smartphone, assess their performance, and provide feedback to improve their real-world usability.
Key Responsibilities: Test AI agents on tasks such as shopping, bookings, bill payments, returns, email follow-ups, travel, and online forms. Approve or stop agents at important decision points. Screen-record every task attempt. Score performance based on completion, quality, user control, satisfaction, and time. Provide clear written feedback on the overall user experience.
Required Qualifications: Must be based in the United States. Bachelor's degree required.2–3 years of professional experience, preferably in engineering, business management, or operations. Regular use of AI tools in daily life. Strong attention to detail, written communication, and documentation skills. An active personal AI assistant account (Plus tier), active Muse and Dots access, a smartphone with screen-recording capability, and a payment card for reimbursable test purchases. No coding or AI technical background is required. Generalists are welcome, and domain experts are preferred. Project Details Kickoff: Project is already active. Onboarding: Within 24–48 hours of selection.
Work Schedule: Flexible, entirely from your phone. Task Duration: Approximately 1.5 hours per run, including evidence and scoring.
Selection Process: Apply and complete the screening questions. Attend a 15–30-minute interview. Complete the profile review. Receive onboarding instructions from the Talent Operations team.
Why Join? Work with AI tools and evaluate real-world performance. Enjoy flexible, phone-based work. Help shape and improve next-generation AI agents.

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this