Generative AI Engineer

Please find the below needs and let me know your interest. Job Title: Generative AI Evaluations Lead | LLM Quality & Human Annotation Location: Oakland, CA Duration: Long term AI/LLM Evaluation + Human Annotation + Project Management + Quality Operations + Strong Communication Key Responsibilities: Lead and coordinate evaluations of Generative AI and LLM outputs to ensure quality, accuracy, consistency, and adherence to evaluation guidelines. Partner with AI, Engineering, Product, and Operations teams to evaluate model responses across multiple use cases. Manage human-in-the-loop evaluation and data annotation workflows, ensuring high-quality and consistent results. Develop and maintain evaluation guidelines, quality standards, scoring rubrics, and annotation processes. Train, coach, and provide feedback to human evaluators to improve annotation accuracy and evaluation consistency. Analyze evaluation results, identify quality trends and gaps, and deliver actionable insights to improve AI model performance. Prioritize evaluation requests and training data needs in collaboration with Product Managers and cross-functional stakeholders. Monitor quality metrics, communicate project status and risks, and support data-driven decision-making. Drive continuous improvement of AI quality management processes and evaluation operations. Required Skills: Experience in project/program management, AI quality operations, or technology product development. Understanding of Generative AI, LLM evaluation, NLP, and human-in-the-loop processes. Experience with data annotation, quality assurance, evaluation frameworks, or evaluator training is highly desirable. Develop and implement a comprehensive AI quality management strategy for TurboTax Excellent communication, stakeholder management, analytical, and organizational skills. Central Point Partners, LLC

#J-18808-Ljbffr