What you will do
- Perform manual testing of web applications to identify bugs, glitches, and inconsistencies;
- Develop, maintain, and execute detailed manual test cases based on product requirements and specifications;
- Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers;
- Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs;
- Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance;
- Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate;
- Conduct manual API testing to validate request and response accuracy, error handling, and integration;
- Use TestRail to create, manage, and track test plans, test cases, and test execution results;
- Use Jenkins CI/CD pipelines;
- Report and document defects clearly using JIRA;
- Participate in daily stand-ups, sprint planning, and other Agile team meetings;
- Review user stories and acceptance criteria to understand the scope of testing;
- Estimate own tasks and report risks to the team lead;
- Collaborate with developers and other QA team members to reproduce issues and verify fixes;
- Maintain and update test documentation as needed.
Must haves
- 4+ years of experience with manual testing;
- Basic understanding of methodologies and SDLC and STLC models;
- Knowledge of testing and quality assurance processes;
- Strong experience with test design activities to cover a set of product requirements;
- Knowledge of test design techniques and testing methodologies;
- Defect analysis skills and the ability to write detailed and useful defect reports;
- Experience with Postman for API testing and browser developer tools;
- Experience with UI testing;
- Experience with TestRail as a test management system;
- Basic understanding of CI/CD processes;
- Intermediate English level.
Nice to haves
- Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles;
- Familiarity with AI and ML concepts, APIs, and frameworks such as OpenAI API, Azure OpenAI, or LangChain;
- Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization;
- Experience with SSO testing;
- Experience in performance or load testing.
We are looking for a Senior QA Engineer to perform manual testing of web applications and validate AI-generated outputs as part of a product team building LLM-integrated features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt–response workflows.
About the role
The benefits of joining us
Professional growth
Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps
Competitive compensation
We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities
A selection of exciting projects
Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands
Flextime
Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.
Your AgileEngine journey starts here
2 min
Tell us about yourself
2 sec
Confirm requirements
30 - 60 min
Pass a short test
5 min
Record a short video
→ Introduce yourself on a video, instead of waiting for an interview
Live interview
Ace the technical interview with our team
→ Schedule a call yourself right away after your video is reviewed
Live interview
Final interview with your team
→ Get to know the team you will be working with
Get an offer
As quick as possible
