Job Description
Job Description Role: AI Agentic Tester
Location: Denver, CO (Zip Code-80221)- REMOTE
Skills: QA/Test Engineering experience
Must have- Experience testing Generative AI, LLM, or AI-based applications.
Keyword:
Skills: Modern Functional Quality Engineering
Experience Required: 2-4 Years
PFB the responsibilities and skills needed:
Functional Testing
- Validate AI agent workflows against business requirements.
- Test end-to-end AI use cases, including multi-agent interactions.
- Verify task completion accuracy and output quality.
- Test agent orchestration, tool calling, and workflow execution.
AI Model Validation
- Evaluate LLM responses for accuracy, relevance, and consistency.
- Test prompt engineering effectiveness and response quality.
- Validate Retrieval-Augmented Generation (RAG) outputs.
- Measure hallucination rates and response reliability.
Required Skills
- 3-8 years of QA/Test Engineering experience.
- Experience testing Generative AI, LLM, or AI-based applications.
- Knowledge of AI Agents, Agentic AI, RAG, and prompt engineering.
- Hands-on experience with API testing (Postman, Swagger, REST APIs).
- Experience with automation frameworks (Selenium, Playwright, PyTest).
- Proficiency in Python.
- Understanding of AI evaluation metrics and testing methodologies
- Knowledge of Azure OpenAI, AWS Bedrock, Gemini, or similar AI platforms