AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability)

Capital One's

Intelligent Foundations and Experiences (IFX)

team builds and deploys proprietary AI platforms that help teams across the company enhance products with responsible, scalable machine learning. In this role, you will design and deliver AI systems focused on

benchmarking, evaluation, explainability , and the foundations needed to run large language model applications in production. As an AI Engineer 4, you will work across the full lifecycle of AI software components, from model training through LLM inference, agents and guardrails, and ongoing monitoring. You'll also help shape the long-term technical vision for foundational AI systems, with an emphasis on reliability, maintainability, and ethical alignment. What you’ll do

Collaborate with engineers, research scientists, technical program managers, and product managers to deliver AI-powered products for associates and customers. Design, develop, test, deploy, and support AI components including

foundation model training ,

large language model inference ,

agents and multi-agent workflows ,

similarity search ,

guardrails ,

model evaluation , experimentation, governance, and observability. Use a broad stack of open source and SaaS AI technologies such as

AWS Ultraclusters ,

Huggingface ,

VectorDBs , and

PyTorch , along with cloud services including AWS, Google Cloud, and Azure. Introduce state-of-the-art optimization techniques to improve performance and production efficiency, focusing on

scalability ,

cost ,

latency , and

throughput . Own end-to-end architecture for complex AI systems, ensuring

maintainability ,

observability , and

ethical alignment . Define and maintain

service-level objectives (SLOs)

for AI reliability, including latency, uptime, and model performance drift. Partner with infrastructure engineering to optimize

GPU/TPU utilization

and accelerate inference pipelines. Lead cross-functional technical reviews for new AI system deployments, supporting security, data governance, and compliance standards. Mentor Principal and Senior Associates on scalable design, performance tuning, and research-to-production translation. What we’re looking for

Education : Bachelor's Degree or Master's degree in Computer Science/AI or related fields. Experience : At least

4 years

developing AI and ML algorithms/technologies, or a Master's degree with at least

2 years

of such experience. Programming : At least

4 years

programming with

Python ,

Go ,

Scala ,

CUDA , or

Java . Tools and technologies

AWS Ultraclusters, Huggingface, VectorDBs, PyTorch AWS, Google Cloud, Azure Python, Go, Scala, CUDA, Java, C++, C#, Golang GPU, TPU Benefits

A comprehensive, competitive, and inclusive set of

health, financial, and other benefits

to support total well-being. Performance-based incentive compensation , which may include cash bonus(es) and/or long term incentives (LTI).

#J-18808-Ljbffr