AI Annotation / Intent Verification Specialist (Entry-Level)
Location:
Remote, based in the Philippines Work Hours: U.S. PST business hours Employment Type: Full-time
Overview:
We are looking for an AI Annotation Specialist to support the evaluation and improvement of a conversational AI system used to automate hotel front desk calls. This role focuses on listening to real guest calls, validating AI intent classification and responses, and helping establish high quality ground truth data that feeds model accuracy, product decisions, and future capability expansion.
You will work closely with a product and AI team to ensure annotations are consistent, reliable, and actionable.
Key Responsibilities:
Listen to and annotate recorded hotel phone calls using internal annotation tools
Validate first level and second level intents predicted by the AI system
Verify the correctness and relevance of AI responses against hotel knowledge bases and documented policies
Flag incorrect answers, missing information, or failure patterns
Identify and propose new intents or sub-intents based on real guest conversations
Follow detailed annotation Standard Operating Procedures to ensure consistency
Log results and insights in shared spreadsheets or internal dashboards
Provide weekly summaries highlighting accuracy, volume, and recurring error patterns
Expected Volume Approximately 500 calls per week
Average annotation time of around 10 minutes per call
Required Skills and Qualifications:
Strong English listening comprehension and attention to detail
Prior experience in data labeling, annotation, quality assurance, or content review
Comfort working with transcripts, audio recordings, and structured guidelines
Proficiency with Google Sheets or similar spreadsheet tools
Ability to follow detailed processes and apply consistent judgment across large volumes of work
Reliable internet connection and ability to work full time hours
Nice to Have:
Familiarity with hospitality, customer service, or call center environments
Experience annotating conversational AI, speech data, or customer support interactions
Basic understanding of AI or machine learning workflows
Tools and Collaboration Internal call playback and transcript tools
Google Sheets and internal dashboards
Slack for day to day communication and weekly syncs
Secure access to systems handling sensitive call data
Why This Role Matters Your work directly impacts how well the AI understands guests, answers questions, and improves over time. High quality annotation is foundational to making the system more accurate, reliable, and useful for hotels around the world.
