R
AI Engineer (Agentic AI / AWS Bedrock)
RIT Solutions, Inc.
2 hours ago
Full-time
On-site
Los Angeles, California, United States
Title:
AI Engineer (Agentic AI / AWS Bedrock) Location:
Los Angeles, CA - locals only Work Arrangement:
Hybrid - 2 days onsite in LA, CA Duration:
6+ month contract-to-hire Experience Range:
6-10 years
Our client is seeking an
AI Engineer
with 6-10 years of experience to design, develop, and deploy intelligent AI agents using Claude Agent SDK and Amazon Bedrock, building production-ready agentic systems with comprehensive observability and evaluation frameworks. Responsibilities & Qualifications
Design, develop, and deploy AI agents using Claude Agent SDK and Python; build intelligent, autonomous workflows capable of executing complex tasks and interacting with external tools, APIs, and knowledge bases. Develop agent orchestration workflows on Amazon Bedrock AgentCore Runtime with error handling, retry mechanisms, and guardrails; integrate agents with enterprise applications, data sources, and foundation models. Implement comprehensive observability and monitoring using Langfuse for LLM and agent tracing, capturing prompts, responses, tool calls, execution metrics, and production performance analytics. Design and implement continuous evaluation frameworks for production AI agent traffic using LLM-as-a-Judge, RAGAS, DeepEval, and custom metrics to detect regressions and ensure quality benchmarks. Build scalable RAG pipelines and vector database solutions (OpenSearch, Pinecone, FAISS, pgvector) for document ingestion, chunking, embedding generation, and knowledge base retrieval optimization. Develop AWS Step Functions workflows integrating Amazon Textract and Amazon Comprehend for intelligent document processing and data transformation at scale. Build and maintain CI/CD pipelines using GitHub Actions, automating agent code builds, testing, deployment, and integrated evaluation workflows for Amazon Bedrock AgentCore Runtime. Implement AWS security best practices including IAM policies, access controls, authentication, CloudWatch monitoring, and cost optimization for production agent systems. Requirements
6-10 years of professional software engineering or AI/ML experience with demonstrated expertise in
Python
and
LLM-based systems . Advanced proficiency with
Claude Agent SDK , agent orchestration, tool calling, and agentic AI frameworks ( LangChain ,
LangGraph ); proven experience deploying AI agents in production environments. Deep expertise with
Amazon Bedrock ,
Bedrock AgentCore Runtime , and AWS services ( Step Functions ,
Textract ,
Comprehend ,
OpenSearch ,
IAM ,
CloudWatch ). Hands-on experience with LLM observability and evaluation tools:
Langfuse ,
LLM-as-a-Judge
frameworks,
RAGAS ,
DeepEval , and custom evaluation metrics. Strong proficiency with
vector databases
and
RAG implementation
( Pinecone ,
OpenSearch ,
FAISS ,
pgvector ); experience with embeddings, semantic search, and retrieval optimization. Expertise in CI/CD and DevOps:
GitHub Actions ,
Git ,
Docker ,
Terraform , Infrastructure as Code, and automated deployment pipelines. Proficiency in prompt engineering,
Model Context Protocol (MCP) , AI governance principles, and AWS architecture best practices for secure, scalable systems. W2 employment only; must be available for hybrid work arrangement.
AI Engineer (Agentic AI / AWS Bedrock) Location:
Los Angeles, CA - locals only Work Arrangement:
Hybrid - 2 days onsite in LA, CA Duration:
6+ month contract-to-hire Experience Range:
6-10 years
Our client is seeking an
AI Engineer
with 6-10 years of experience to design, develop, and deploy intelligent AI agents using Claude Agent SDK and Amazon Bedrock, building production-ready agentic systems with comprehensive observability and evaluation frameworks. Responsibilities & Qualifications
Design, develop, and deploy AI agents using Claude Agent SDK and Python; build intelligent, autonomous workflows capable of executing complex tasks and interacting with external tools, APIs, and knowledge bases. Develop agent orchestration workflows on Amazon Bedrock AgentCore Runtime with error handling, retry mechanisms, and guardrails; integrate agents with enterprise applications, data sources, and foundation models. Implement comprehensive observability and monitoring using Langfuse for LLM and agent tracing, capturing prompts, responses, tool calls, execution metrics, and production performance analytics. Design and implement continuous evaluation frameworks for production AI agent traffic using LLM-as-a-Judge, RAGAS, DeepEval, and custom metrics to detect regressions and ensure quality benchmarks. Build scalable RAG pipelines and vector database solutions (OpenSearch, Pinecone, FAISS, pgvector) for document ingestion, chunking, embedding generation, and knowledge base retrieval optimization. Develop AWS Step Functions workflows integrating Amazon Textract and Amazon Comprehend for intelligent document processing and data transformation at scale. Build and maintain CI/CD pipelines using GitHub Actions, automating agent code builds, testing, deployment, and integrated evaluation workflows for Amazon Bedrock AgentCore Runtime. Implement AWS security best practices including IAM policies, access controls, authentication, CloudWatch monitoring, and cost optimization for production agent systems. Requirements
6-10 years of professional software engineering or AI/ML experience with demonstrated expertise in
Python
and
LLM-based systems . Advanced proficiency with
Claude Agent SDK , agent orchestration, tool calling, and agentic AI frameworks ( LangChain ,
LangGraph ); proven experience deploying AI agents in production environments. Deep expertise with
Amazon Bedrock ,
Bedrock AgentCore Runtime , and AWS services ( Step Functions ,
Textract ,
Comprehend ,
OpenSearch ,
IAM ,
CloudWatch ). Hands-on experience with LLM observability and evaluation tools:
Langfuse ,
LLM-as-a-Judge
frameworks,
RAGAS ,
DeepEval , and custom evaluation metrics. Strong proficiency with
vector databases
and
RAG implementation
( Pinecone ,
OpenSearch ,
FAISS ,
pgvector ); experience with embeddings, semantic search, and retrieval optimization. Expertise in CI/CD and DevOps:
GitHub Actions ,
Git ,
Docker ,
Terraform , Infrastructure as Code, and automated deployment pipelines. Proficiency in prompt engineering,
Model Context Protocol (MCP) , AI governance principles, and AWS architecture best practices for secure, scalable systems. W2 employment only; must be available for hybrid work arrangement.