Job Description

SDET with AI


Collaborate with business stakeholders to identify and define use cases

Design develop and execute automated tests using Python to ensure the functionality and performance of the product

Conduct manual testing where necessary to complement automated tests

Write complex SQL queries for test data generation and validation

Test and validate the product developed by the developers ensuring it meets the specified requirements

Stay informed and proficient in GenAI evaluation frameworks applying them effectively to assess the product

Perform rigorous testing of machine learning ML and deep learning DL components

Knowledge of Synthetic data generation techniques

Demonstrate in depth knowledge of evaluation based frameworks such as RAGAS Retrieval Augmented Generation for Automated Systems and a basic understanding of how RAG Retrieval Augmented Generation works

Define and implement testing strategies

Exhibit excellent communication and documentation skills to clearly report issues document test cases and articulate the results to technical and non technical stakeholders

Apply analytical skills to troubleshoot and resolve complex problems ensuring the highest quality standards are met



Skills and Qualifications

Proven experience in testing AI ML based solutions particularly generative AI GenAI

Proficiency in automated testing with Python

Familiarity with manual testing processes

Strong knowledge of SQL for data manipulation and testing purposes

Demonstrated experience with GenAI evaluation frameworks and testing of ML DL solutions

Expertise in prompt engineering for LLMs including question generation and evaluation metrics

Comprehensive knowledge of RAGAS and RAG frameworks

Experience with evaluating chatbot use cases and ensuring their effectiveness and relevance

Exceptional communication and documentation abilities

Strong analytical and problem solving skills

About Virtusa



Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 30,000 people globally that cares about your growth -- one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.



Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.



Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.