Full Stack Developer - Agentic Test Automation

Full Stack Developer - Agentic Test Automation - onsite location in McLean, VA as contractors.

Local candidates are highly preferred.

Hourly rate (without benefit): $55-75/hr

Must Have Qualifications: Must have 5+ years of experience and a strong AI development background. Must have hands on experience building agentic workflows, automating testing using AI, and working with tools such as GitHub, Copilot, and Claude. 

Interview Information: 

Rounds: 2 rounds 

Duration: 30-60 mins 

Interview Type: 1st round – virtual | 2nd  round – onsite 

JOB DESCRIPTION 

1) Agentic test automation foundation (reusable patterns + reference implementations) 

• Design and implement agentic testing patterns that can be adopted by multiple 

Underwriting teams (and later other domains). 

• Create reference implementations (sample repos / templates) demonstrating: 

o Test generation assistance (from requirements, APIs, contracts, schemas) 

o Test maintenance assistance (auto-updating selectors/contracts, flaky test triage) o Failure analysis assistance (root cause suggestions, log correlation, defect drafting) 

• Establish a standard architecture for test code organization, tagging, data management, and execution across UI + API + service layers. 

2) Coverage standards, templates, and governance 

• Define and publish coverage standards (what “good” looks like) including:

o Minimum coverage expectations by service/component 

o Test type mix (unit vs API vs UI vs contract vs integration) 

o Risk-based prioritization and traceability to requirements 

• Provide templates usable across teams: 

o Test plan templates 

o Test case/spec templates (Gherkin-style or equivalent) 

o Definition of Ready / Definition of Done quality checklists 

• Create a scalable tagging/metadata strategy (e.g., feature, service, risk, priority, data 

sensitivity) to support reporting and quality gates. 

3) GenAI-assisted reporting and quality insights across microservices 

• Build automated reporting that aggregates test + service data across multiple 

microservices, such as: 

o Test execution results (Karate/Playwright + CI runs) 

o Service health signals (logs/metrics/traces if available) 

o Defect signals (issue tracker metadata if available) 

• Generate GenAI-driven summaries: 

o Release readiness narratives 

o Failure clustering and trend analysis 

o “What changed?” insights (commit/PR correlation) 

• Produce outputs consumable by engineering leadership and teams (dashboards,

markdown summaries in PRs, artifacts in CI).

4) “Quality gates” via agents 

• Build automated review agents that evaluate user stories/requirements for minimum 

required clarity and data before development/testing starts: 

o Required fields present (acceptance criteria, testable outcomes, 

data needs, dependencies)o Ambiguity detection and missing edge cases 

o Data/privacy considerations and environment needs 

• Integrate gates into workflow (PR checks, issue templates, GitHub Actions) to reduce churn and rework. 

Required Technical Skills (must-have) GenAI 

/ LLM + agentic development 

• Hands-on experience building LLM-powered agents (tool-using, multi-step reasoning, guardrails). 

• Experience with prompting patterns, structured outputs (JSON schemas), evaluation, and reducing hallucinations. 

• Ability to design agent workflows for: 

o Test generation/augmentation 

o Requirements review and completeness validation 

o Report generation and summarization 

GitHub platform + GHCP (Copilot) for engineering workflows 

• Strong proficiency with GitHub Copilot in day-to-day development. 

• Deep experience with GitHub platform capabilities: 

o GitHub Actions (CI/CD pipelines, reusable workflows, composite actions) 

o PR checks, branch protections, CODEOWNERS, templates o Automation via 

GitHub APIs/webhooks (as needed) 

Test automation engineering (framework expertise) 

• Advanced experience designing and implementing automation with: 

o Karate (API testing, contract-like checks, data-driven 

testing, mocks) 

o Playwright (UI automation, selectors strategy, 

parallelization, trace/video artifacts) 

• Strong understanding of test design and coverage:

o Happy path scenarios 

o Negative/validation scenarios 

O Edge/boundary scenarios 

o Data setup/teardown strategies and test isolation Cross

service reporting and data aggregation 

• Proven ability to aggregate and normalize results from multiple microservices and multiple pipelines. 

• Experience producing actionable automated reports (trend analysis, failure clustering, service correlation). 

Automated requirements review agents 

• Experience implementing automated checks that validate: 

o Acceptance criteria completeness 

o Required test data and environment dependencies 

o Non-functional requirements (performance, security, observability) when 

applicable 

Deliverables / What success looks like (for the posting) 

• A reusable agentic testing automation kit adopted by multiple teams. 

• Published coverage standards + templates and onboarding documentation. 

• A working GenAI-assisted reporting pipeline aggregating results across microservices. 

• Automated quality gates integrated into GitHub workflows that measurably reduce story churn