-
Summary
Agentic Judge
Location: Remote ( local to Seattle, WA )
Duration: 06 Months
Description:- The Artificial Intelligence team is harnessing the power of Generative Artificial Intelligence to build services and product experiences for a seamless and convenient real estate transaction experience.
- The Artificial Intelligence Platform team is building platform services that enable usage of Large Language Models (LLMs) and multimodal models across client businesses.
- Will work a team of real-estate experts and contribute to our growing set of Agentic projects.
- Combines expertise in the real-estate process stages with a deep curiosity around Artificial Intelligence-powered experiences.
About the role:- The Annotation Judge plays a crucial role in fine-tuning and calibrating AI/ML processes that handle verbal interactions, between home buyers and real estate agents for example.
- Works directly with experienced Operations Managers, ML Engineers, Data Scientist, Software Developers and Natural Language experts.
- Responsible for ensuring the quality of a number of possible elements related to our projects. Projects such as reviewing the accuracy and humanlike traits of automated chatbots answers, to the classification of elements of a conversation annotated automatically or the review of search results based on understanding of complex users queries.
- Plays an essential role in defining clear guidelines which may be used by non-native English speakers. Attention to detail and natural curiosity for language subtleties is essential for the long term scaling of our solutions.
- Review and/or evaluate annotated data to ensure it meets quality standards and project requirements.
- Provide feedback and guidance to engineers to improve accuracy.
- Collaborate with engineers and data scientists to optimize model performance based on annotated data.
- Develop and maintain annotation guidelines and best practices.
- Provide training data to engineers to establish Ground Truth data.
- Provide input/examples on Training content.
- Identify gaps in training content and suggest improvements.
MINIMUM QUALIFICATIONS- Minimum of 3 years of active experience as a licensed real estate agent or broker, demonstrating a deep understanding of the real estate transaction process.
- Demonstrated deep interest and curiosity in Artificial Intelligence, including experience reviewing or evaluating AI-generated content.
- Bachelor's degree in English, Linguistics or a related field.
- Experience working on Natural Language Processing projects with a focus on transcriptions, localization, translation.
- Excellent attention to detail and analytical skills.
- Ability to work effectively in a collaborative team environment.
- Experience with tools and software used for Natural Language Processing annotation is a plus.
- Experience with copywriting, User Experience authoring, content development around voice and brand guidelines.
- Experience with Machine-generated Speech evaluation, Search Results Evaluation, Search Query Categorization
- Experience following or managing complex processes, and working both independently and with a team to achieve a goal.
-
Job Description
Agentic Judge
Location: Remote ( local to Seattle, WA )
Duration: 06 Months
Description:- The Artificial Intelligence team is harnessing the power of Generative Artificial Intelligence to build services and product experiences for a seamless and convenient real estate transaction experience.
- The Artificial Intelligence Platform team is building platform services that enable usage of Large Language Models (LLMs) and multimodal models across client businesses.
- Will work a team of real-estate experts and contribute to our growing set of Agentic projects.
- Combines expertise in the real-estate process stages with a deep curiosity around Artificial Intelligence-powered experiences.
About the role:- The Annotation Judge plays a crucial role in fine-tuning and calibrating AI/ML processes that handle verbal interactions, between home buyers and real estate agents for example.
- Works directly with experienced Operations Managers, ML Engineers, Data Scientist, Software Developers and Natural Language experts.
- Responsible for ensuring the quality of a number of possible elements related to our projects. Projects such as reviewing the accuracy and humanlike traits of automated chatbots answers, to the classification of elements of a conversation annotated automatically or the review of search results based on understanding of complex users queries.
- Plays an essential role in defining clear guidelines which may be used by non-native English speakers. Attention to detail and natural curiosity for language subtleties is essential for the long term scaling of our solutions.
- Review and/or evaluate annotated data to ensure it meets quality standards and project requirements.
- Provide feedback and guidance to engineers to improve accuracy.
- Collaborate with engineers and data scientists to optimize model performance based on annotated data.
- Develop and maintain annotation guidelines and best practices.
- Provide training data to engineers to establish Ground Truth data.
- Provide input/examples on Training content.
- Identify gaps in training content and suggest improvements.
MINIMUM QUALIFICATIONS- Minimum of 3 years of active experience as a licensed real estate agent or broker, demonstrating a deep understanding of the real estate transaction process.
- Demonstrated deep interest and curiosity in Artificial Intelligence, including experience reviewing or evaluating AI-generated content.
- Bachelor's degree in English, Linguistics or a related field.
- Experience working on Natural Language Processing projects with a focus on transcriptions, localization, translation.
- Excellent attention to detail and analytical skills.
- Ability to work effectively in a collaborative team environment.
- Experience with tools and software used for Natural Language Processing annotation is a plus.
- Experience with copywriting, User Experience authoring, content development around voice and brand guidelines.
- Experience with Machine-generated Speech evaluation, Search Results Evaluation, Search Query Categorization
- Experience following or managing complex processes, and working both independently and with a team to achieve a goal.
-
ABOUT THE COMPANY
-
-
Government Careers
Show moreGovernment jobs offer stability, competitive benefits, and the chance to make a meaningful impact on your community and country.
Whether you’re starting your career or seeking new opportunities, these roles provide pathways for growth, security, and service.
Explore positions across a wide range of fields and take the first step toward a rewarding future in public service.
-