Collaborate with Applied AI Engineers and Data Scientists to design and implement procedures to validate AI products on LLMs for accuracy, benchmarking, and functionality.
Maintain detailed records of all validation activities, including test methods, results, and any issues encountered
Evaluate test data to determine whether AI systems or processes have met validation criteria and identify areas for improvement
Suggest tools and methodologies for efficient validation and benchmarking of AI products.
Automate validation steps / tasks to improve efficiency in validation and benchmarking.
Assist in creating sample datasets for benchmarking and functionality tests.
Ensure the AI products meet the required standards and specifications.
Provide insights and recommendations for improving AI product performance.
Keep up-to-date with evolving industry standards, technological advancements, and best practices and new trends in AI validation
Proven experience delivering high-quality outcomes in fast-paced, agile environments.
Qualifications:
Bachelors degree in computer science, Engineering, Data Science, or a related field.
4 to 8 years of experience in AI Engineering, data science, AI validation or a related field.
Strong Experience with validation and benchmarking of AI products.
Good understanding of AI and machine learning concepts, particularly LLMs.
Familiarity with tools and methodologies for AI validation and benchmarking.
Excellent documentation and communication skills.
Ability to work collaboratively in a team environment.
Strong problem-solving skills and attention to detail.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Quality AssuranceRole Category: Quality Assurance - OtherRole: Quality Assurance - OtherEmployement Type: Full time