JOB DESCRIPTION
We are seeking a Mid–Senior AI Engineer to build, integrate, and evaluate agentic AI and RAG systems for an AI testing and evaluation platform. The role focuses on production-grade AI application engineering, agent reliability, safety, guardrails, and AI risk evaluation.
You will collaborate with product and customer delivery teams to improve AI evaluation capabilities and support complex customer use cases.
- Design, build, and integrate agentic AI systems into the AI evaluation platform.
- Develop agent harnesses, guardrails, and mechanisms to assess AI agent reliability and safety.
- Build and stress-test agentic AI and Retrieval-Augmented Generation (RAG) systems.
- Evaluate agent behavior, identify failure modes, and improve system robustness.
- Generate synthetic data to support AI testing and evaluation.
- Extend evaluation capabilities to cover new AI risk areas and use cases.
- Collaborate with product teams to develop evaluation methods and testing approaches.
- Work with customer technical teams to define evaluation requirements and implement AI solutions in customer environments where needed.
REQUIRED SKILLS AND EXPERIENCE
- 2-3+ years of experience in AI/ML Engineering
- Strong hands-on experience building agentic AI systems, including agent harnesses and guardrails.
- Strong Python programming skills.
- Proven experience in production-grade LLM application engineering.
- Hands-on experience designing, building, or testing agentic AI and RAG systems.
- Understanding of AI reliability, safety, testing methodologies, and failure-mode handling.
- Experience generating synthetic data for AI evaluation.
- Clear spoken and written English (B2+).
PREFERRED QUALIFICATIONS
- Experience with AI governance and security frameworks, such as AI Verify, NIST AI RMF, or OWASP Top 10 for LLM Applications.
- Previous experience in AI evaluation, AI red-teaming, or AI assurance.
- Familiarity with computer vision.
- Experience delivering AI solutions in customer environments or collaborating directly with customer technical teams.
