C
Distinguished AI Engineer
Compunnel
3 hours ago
Full-time
On-site
Jersey City, New Jersey, United States
div:has([data-free-thinking-preview-answer=true])+:is(.text-message,.relative:has(>.text-message))]:-mt-2 grow">
Job Summary The Distinguished AI Engineer will provide senior individual-contributor technical leadership for an enterprise AI platform, focusing on solving complex AI platform engineering challenges and developing reusable, secure, reliable, observable, and cost-efficient capabilities. The role will drive the technical evolution of shared AI platform capabilities, including AI gateways, model access and serving, Retrieval-Augmented Generation (RAG), agent and tool execution, orchestration, evaluation, observability, and LLMOps/MLOps. The position will lead through deep technical expertise, engineering assets, and influence rather than direct people management, while supporting enterprise AI and Generative AI solutions at scale.
Key Responsibilities • Lead the design and engineering evolution of AI platform services for model access, routing, serving, RAG, agents, orchestration, evaluation, and observability. • Solve complex engineering trade-offs across capability, latency, throughput, resiliency, security, data isolation, portability, and cost. • Lead critical technical spikes, prototypes, reference implementations, deep design reviews, and resolution of major platform or production issues. • Establish measurable engineering expectations for availability, recovery, performance, capacity, telemetry, release safety, evaluation coverage, and inference cost. • Build or sponsor reusable engineering assets, including APIs, SDKs, Terraform/IaC modules, deployment patterns, CI/CD templates, dashboards, evaluation tooling, and developer guidance. • Drive production excellence through observability, traceability, controlled releases, rollback capabilities, incident learning, capacity planning, and cost optimization. • Engineer security and AI controls into platform capabilities, including identity and access controls, authorization-aware retrieval, secure tool execution, prompt-injection defenses, data protection, logging, and audit evidence. • Partner with cybersecurity, risk, compliance, legal, audit, architecture, product, and business teams to translate enterprise requirements into usable technical controls and delivery patterns. • Assess emerging AI technologies through practical technical evaluations and recommend adoption based on value, maturity, risk, operational fit, and total cost of ownership. • Mentor senior engineers and drive adoption of enterprise AI platform patterns across teams that do not report directly to the role.
Required Qualifications • 10+ years of progressive experience in software engineering, distributed systems, cloud/platform engineering, AI/ML infrastructure, or related technical domains, including substantial experience leading complex systems across multiple teams. • Demonstrated experience building, scaling, transforming, or supporting production platforms or critical technical systems used by multiple engineering teams, products, or business domains. • Deep hands-on expertise in several of the following areas: LLM platforms, model serving, inference optimization, AI gateways, model routing, RAG, embeddings/vector search, agent frameworks, tool execution, AI orchestration, LLMOps/MLOps, evaluation systems, AI observability, or AI security controls. • Strong engineering foundation in distributed systems, API and platform design, cloud-native architecture, containers/Kubernetes, networking, IAM, and secrets management. • Experience designing and engineering reliable, scalable, secure, and production-ready AI platform capabilities. • Strong technical judgment and ability to evaluate complex trade-offs involving performance, resiliency, security, scalability, portability, and cost. • Ability to lead complex technical initiatives through architecture, prototyping, implementation, production deployment, and operational support. • Strong collaboration and communication skills with technical and non-technical stakeholders across cybersecurity, risk, compliance, legal, audit, architecture, product, and business functions.
Job Summary The Distinguished AI Engineer will provide senior individual-contributor technical leadership for an enterprise AI platform, focusing on solving complex AI platform engineering challenges and developing reusable, secure, reliable, observable, and cost-efficient capabilities. The role will drive the technical evolution of shared AI platform capabilities, including AI gateways, model access and serving, Retrieval-Augmented Generation (RAG), agent and tool execution, orchestration, evaluation, observability, and LLMOps/MLOps. The position will lead through deep technical expertise, engineering assets, and influence rather than direct people management, while supporting enterprise AI and Generative AI solutions at scale.
Key Responsibilities • Lead the design and engineering evolution of AI platform services for model access, routing, serving, RAG, agents, orchestration, evaluation, and observability. • Solve complex engineering trade-offs across capability, latency, throughput, resiliency, security, data isolation, portability, and cost. • Lead critical technical spikes, prototypes, reference implementations, deep design reviews, and resolution of major platform or production issues. • Establish measurable engineering expectations for availability, recovery, performance, capacity, telemetry, release safety, evaluation coverage, and inference cost. • Build or sponsor reusable engineering assets, including APIs, SDKs, Terraform/IaC modules, deployment patterns, CI/CD templates, dashboards, evaluation tooling, and developer guidance. • Drive production excellence through observability, traceability, controlled releases, rollback capabilities, incident learning, capacity planning, and cost optimization. • Engineer security and AI controls into platform capabilities, including identity and access controls, authorization-aware retrieval, secure tool execution, prompt-injection defenses, data protection, logging, and audit evidence. • Partner with cybersecurity, risk, compliance, legal, audit, architecture, product, and business teams to translate enterprise requirements into usable technical controls and delivery patterns. • Assess emerging AI technologies through practical technical evaluations and recommend adoption based on value, maturity, risk, operational fit, and total cost of ownership. • Mentor senior engineers and drive adoption of enterprise AI platform patterns across teams that do not report directly to the role.
Required Qualifications • 10+ years of progressive experience in software engineering, distributed systems, cloud/platform engineering, AI/ML infrastructure, or related technical domains, including substantial experience leading complex systems across multiple teams. • Demonstrated experience building, scaling, transforming, or supporting production platforms or critical technical systems used by multiple engineering teams, products, or business domains. • Deep hands-on expertise in several of the following areas: LLM platforms, model serving, inference optimization, AI gateways, model routing, RAG, embeddings/vector search, agent frameworks, tool execution, AI orchestration, LLMOps/MLOps, evaluation systems, AI observability, or AI security controls. • Strong engineering foundation in distributed systems, API and platform design, cloud-native architecture, containers/Kubernetes, networking, IAM, and secrets management. • Experience designing and engineering reliable, scalable, secure, and production-ready AI platform capabilities. • Strong technical judgment and ability to evaluate complex trade-offs involving performance, resiliency, security, scalability, portability, and cost. • Ability to lead complex technical initiatives through architecture, prototyping, implementation, production deployment, and operational support. • Strong collaboration and communication skills with technical and non-technical stakeholders across cybersecurity, risk, compliance, legal, audit, architecture, product, and business functions.