Berkeley Heights, NJ / Alpharetta, GA (5 Days onsite)
Employment Type:
long term contract
Job description
Job Summary:
Required Qualifications
Must Have:
Strong software engineering fundamentals and proficiency in Python plus Java Go TypeScript is a strong plus
Experience of working with Codex.
Proven experience building LLM powered applications in production tool calling function, calling structured outputs retrieval and evaluation.
Experience designing distributed systems and APIs REST, RPC plus event driven patterns Kafka, SQS, Pub Sub.
Solid understanding of data engineering basics SQL data modeling feature engineering and data quality
Handson knowledge of cloud platforms AWS or Azure or GCP containers, Docker and orchestration Kubernetes preferred.
Ability to write clean testable secure code comfortable with code reviews and engineering rigor
Experience with multiagent systems planning verification and autonomous workflow execution
Experience with vector databases hybrid search and knowledge graphs
Familiarity with model evaluation offline evals golden datasets adversarial testing regression harnesses and AB testing.
Technical Skills:
Agent frameworks Lang Graph Semantic Kernel similar orchestration frameworks or equivalent custom implementations
RAG tooling embedding pipelines hybrid retrieval reranking chunking strategies citation provenance
Observability Open Telemetry structured logging dashboards ing
Data systems OLTP analytics warehouses lakes streaming pipelines feature stores optional
Testing unit integration tests for tools replay tests for agent traces eval harnesses for LLM outputs
Key Responsibilities:
1. Agentic AI System Design Engineering
Design and implement agent architectures planner executor tool using agents multiagent orchestration reflection evaluation loops.
Build tooling integrations for agents merchant systems underwriting platforms transaction stores risk engines CRM case tools knowledge bases and workflow engines.
Implement robust state management session memory task plans provenance traceability and replay ability of agent actions
2. LLM RAG Engineering for Payments Workloads
Develop RAG pipelines over policies SOPs card network rules underwriting guidelines dispute playbooks and merchant agreements.
Apply prompt and system design structured output patterns and schema validation for deterministic agent behaviour.
Optimize for latency cost and reliability using caching model routing and evaluation driven prompt iteration.
3. ML Decisioning Integration
Combine LLM agents with classical ML models fraud scoring anomaly detection risk scoring and rules engines.
Build feedback loops from outcomes chargeback win rate false positives approval uplift to continuously improve models and agent strategies.
4. Safety Compliance and Responsible AI
Implement guardrails PII handling policy enforcement prompt injection defences tool per missioning rate limiting and safe failover.
Ensure auditability why an agent took action evidence used and human approval where required humanintheloop.
5. Productionization MLOps LLMOps
Build CICD for agent services evaluation suites telemetry drift detection and incident response playbooks.
Instrument agent behavior using tracing spans structured logs and metrics task success tool errors hallucination indicators
6. Collaboration Leadership
Partner with Product Risk Ops Underwriting Compliance and Engineering to convert business problems into deployable AI solutions.
Mentor engineers set standards for agent design patterns testing and production readiness.