T
Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years
TMS LLC
3 hours ago
Full-time
On-site
Los Angeles, California, United States
Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years
Contract Job Summary We are seeking a hands‑on AI Engineer with expertise in Python, Agentic AI, AWS Bedrock AgentCore, and LLM evaluation to build, deploy, monitor, and continuously improve production‑grade AI agents. Key Responsibilities Develop AI agents using Python and Claude Agent SDK. Deploy agents using AWS Bedrock AgentCore Runtime. Design golden datasets and implement LLM-as-a-Judge evaluations. Integrate automated evaluations into CI/CD pipelines as quality gates. Implement Langfuse/OpenTelemetry for AI observability and tracing. Evaluate agent accuracy, task completion, tool execution, and performance. Support RAG pipelines and AWS data processing workflows. Required Skills Python and Agentic AI development. AWS Bedrock AgentCore Evaluations / Bedrock evaluation. Golden-dataset design and curation. LLM-as-a-Judge design and calibration against human labels. Automated evaluation and CI gate integration. Langfuse and/or OpenTelemetry. Good to Have RAGAS or equivalent evaluation frameworks. Trajectory evaluation and agentic red‑teaming. Bedrock Guardrails, Strands Agents SDK, and GitHub Actions. Qualifications 6-8+ years of software engineering, cloud engineering, or AI/ML experience. Hands‑on experience with production AI agents and continuous evaluation. Strong problem‑solving and troubleshooting skills. All your information will be kept confidential according to EEO guidelines.
#J-18808-Ljbffr
Contract Job Summary We are seeking a hands‑on AI Engineer with expertise in Python, Agentic AI, AWS Bedrock AgentCore, and LLM evaluation to build, deploy, monitor, and continuously improve production‑grade AI agents. Key Responsibilities Develop AI agents using Python and Claude Agent SDK. Deploy agents using AWS Bedrock AgentCore Runtime. Design golden datasets and implement LLM-as-a-Judge evaluations. Integrate automated evaluations into CI/CD pipelines as quality gates. Implement Langfuse/OpenTelemetry for AI observability and tracing. Evaluate agent accuracy, task completion, tool execution, and performance. Support RAG pipelines and AWS data processing workflows. Required Skills Python and Agentic AI development. AWS Bedrock AgentCore Evaluations / Bedrock evaluation. Golden-dataset design and curation. LLM-as-a-Judge design and calibration against human labels. Automated evaluation and CI gate integration. Langfuse and/or OpenTelemetry. Good to Have RAGAS or equivalent evaluation frameworks. Trajectory evaluation and agentic red‑teaming. Bedrock Guardrails, Strands Agents SDK, and GitHub Actions. Qualifications 6-8+ years of software engineering, cloud engineering, or AI/ML experience. Hands‑on experience with production AI agents and continuous evaluation. Strong problem‑solving and troubleshooting skills. All your information will be kept confidential according to EEO guidelines.
#J-18808-Ljbffr