Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) job opportunity at Resillion.

Home Jobs In United Kingdom Tag: Job Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) position at Resillion


Date2026-05-05T13:22:26.836Z bot
Resillion Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid)
Experience: 3-years
Pattern: Contract
Country:
apply Apply Now
Salary:
Status:

Job

Copy Link Report
degreeGeneral
loacation Glasgow, Scotland, United Kingdom
loacation Glasgow, Scotl..........United Kingdom
Auto GPT Summarize Enabled
Job DescriptionSenior Automation Tester (Python) – Agentic AI & LLM Systems Location: Glasgow (Hybrid preferred but not essential) Start Date: Immediate Right to Work: Must have full UK working rights (no sponsorship available due to start dates) Additional Requirement: Must have 3 years of UK address history due to immediate onboarding Programme: AI / Agentic AI Seniority: Senior Automation Engineer OverviewWe are seeking a Senior Automation Engineer / SDET with strong Python skills and hands‑on experience testing AI systems, including LLM‑driven applications and agentic AI workflows. You will design and build a scalable automation frameworks, implement modern AI testing strategies, and ensure the quality and reliability of next‑generation conversational and agent‑based AI products.This role is ideal for someone who is passionate about automation engineering and has a solid understanding of how AI/LLM systems behave in real‑world scenarios.Key ResponsibilitiesDevelop and maintain a Python‑based automation framework for validating AI agents (chatbots, voicebots, agentic workflows).Design and execute automated tests for:LLM responsesConversational flowsRetrieval‑augmented generation (RAG)End‑to‑end agentic scenariosApply modern AI testing strategies and evaluation methods (e.g., DeepEval, RAGAS — beneficial but not essential).Integrate automated tests into CI/CD pipelines (Jenkins, GitHub Actions, Azure Repos).Build and manage batch and scheduled executions using Docker, Kubernetes, cron jobs, or similar tooling.Manage code repositories using Git, including branching strategies, version control, and collaborative workflows.·  Collaborate closely with engineering, AI/ML, and product teams to ensure high‑quality delivery of AI‑powered features.Required Skills & ExperienceStrong Python development skills, particularly for automation (PyTest preferred).Solid background in test automation and framework design.Experience testing or validating AI/LLM‑based systems (agentic workflows, conversational AI, RAG pipelines, etc.).Understanding of AI evaluation concepts — familiarity with tools like DeepEval or RAGAS is a bonus, not a requirement.Experience implementing CI/CD pipelines and automated batch executions.Strong understanding of source code management, Git workflows, and collaborative engineering practices.Ability to design structured, scalable, and maintainable automation solutions.Preferred QualitiesQuick learner with a proactive, ownership‑driven mindset.Strong problem‑solving skills and ability to work in fast‑moving AI environments.Comfortable collaborating with cross‑functional teams (AI/ML, engineering, product).Passion for AI technologies and modern testing methodologies. 

Other Ai Matches

Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) Applicants are expected to have a solid experience in handling Job related tasks
Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) Applicants are expected to have a solid experience in handling Job related tasks
Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) Applicants are expected to have a solid experience in handling Job related tasks
Interim Head of Testing - Dublin (Hybrid) - IMMEDIATE START - Contract - Duration (4 to 5 months) - Utilities Sector Experience highly advantageous  Applicants are expected to have a solid experience in handling Job related tasks