F
AI Engineer
Falcon Smart IT (FalconSmartIT)
5 hours ago
Full-time
On-site
Los Angeles, California, United States
AI Developer will build and operate AI-enabled applications for customer experiences, employee productivity, and operations. Use cases may include conversational support, knowledge assistance, search and discovery, summarization, classification, decision support, workflow automation, and content/metadata operations.
This is a production engineering role. Success requires strong software fundamentals, disciplined evaluation, secure enterprise integration, and ownership of quality, latency, cost, observability, and supportability throughout the application lifecycle.
Key responsibilities
? Build production applications using large language models, smaller task-specific models, retrieval-augmented generation, tool/function calling, workflow orchestration, and deterministic business logic where appropriate.
? Develop secure APIs, services, adapters, and event-driven integrations for digital channels, customer-care platforms, enterprise knowledge, billing and entitlement services, content/metadata systems, and internal workflows.
? Implement authorization-aware tool use, input validation, idempotency, timeouts, retries, fallback behaviour, circuit breakers, and human escalation paths.
? Choose prompts, retrieval, rules, conventional machine learning, or fine-tuning based on evidence rather than defaulting every problem to a large model.
Retrieval, data, and grounding:
? Build ingestion, chunking, metadata, indexing, retrieval, reranking, citation, freshness, and deletion workflows for enterprise knowledge and approved content sources.
? Preserve source permissions and customer/data boundaries throughout retrieval and generation; prevent unauthorized cross-user, cross-account, or cross-domain disclosure.
? Partner with Data Engineering and domain owners on data quality, system-of-record alignment, lineage, and feedback loops. Evaluation and quality engineering
? Create representative evaluation datasets and automated test suites for groundedness, relevance, correctness, task completion, refusal behaviour, safety, robustness, latency, and cost.
? Run regression testing across prompt, model, retrieval, tool, and policy changes; analyse failure modes and improve the system using trace-based evidence.
? Instrument online quality and business metrics, support controlled experiments, and incorporate human review for higher-risk or lower-confidence outcomes. Production operations and MLOps
? Build CI/CD pipelines for code, configuration, prompts, evaluation assets, and model or index changes across separated development, test, and production environments.
? Implement structured logging, tracing, token and infrastructure cost monitoring, model/provider health checks, alerting, dashboards, and operational runbooks.
? Optimize throughput, latency, reliability, and cost using caching, batching, routing, prompt/context management, and appropriately sized models.
? Participate in production support, incident response, root-cause analysis, and continuous improvement.
Security and responsible implementation:
? Implement controls for prompt injection, jailbreak attempts, unsafe tool use, data leakage, malicious content, model abuse, and dependency/supply-chain risk.
? Apply DIRECTV requirements for PII and payment-card data, identity and access, secrets management, retention, content rights, audit logging, and approved model/provider use.
? Contribute reusable components to the AI control plane, including policy enforcement, prompt/model configuration, evaluation hooks, telemetry, and kill-switch or rollback mechanisms.
? Work with Product Managers, UX, Solution Architects, AI Architects, Data Engineers, Cybersecurity, Quality Engineering, and Operations to deliver testable user outcomes.
? Write maintainable code, automated tests, interface contracts, technical documentation, deployment guides, and operational runbooks; participate in code and design reviews.
Required qualifications:
Typically, 10+ years in professional software engineering, including meaningful hands-on experience delivering AI, machine-learning, search, NLP, or data-intensive applications to production; equivalent experience is welcome.
Hands-on experience with LLM APIs, prompt and context design, RAG, embedding/search systems, structured outputs, tool/function calling, and automated evaluation.
Experience with SQL and document/search stores, containers, CI/CD, source control, cloud services, and observability practices.
Strong software engineering habits: modular design, automated testing, secure coding, peer review, performance troubleshooting, and production ownership.
Ability to explain model limitations and engineering trade-offs to technical and nontechnical partners.
Bachelor's degree in computer science, engineering, data science, or a related field, or equivalent practical experience.
#J-18808-Ljbffr
This is a production engineering role. Success requires strong software fundamentals, disciplined evaluation, secure enterprise integration, and ownership of quality, latency, cost, observability, and supportability throughout the application lifecycle.
Key responsibilities
? Build production applications using large language models, smaller task-specific models, retrieval-augmented generation, tool/function calling, workflow orchestration, and deterministic business logic where appropriate.
? Develop secure APIs, services, adapters, and event-driven integrations for digital channels, customer-care platforms, enterprise knowledge, billing and entitlement services, content/metadata systems, and internal workflows.
? Implement authorization-aware tool use, input validation, idempotency, timeouts, retries, fallback behaviour, circuit breakers, and human escalation paths.
? Choose prompts, retrieval, rules, conventional machine learning, or fine-tuning based on evidence rather than defaulting every problem to a large model.
Retrieval, data, and grounding:
? Build ingestion, chunking, metadata, indexing, retrieval, reranking, citation, freshness, and deletion workflows for enterprise knowledge and approved content sources.
? Preserve source permissions and customer/data boundaries throughout retrieval and generation; prevent unauthorized cross-user, cross-account, or cross-domain disclosure.
? Partner with Data Engineering and domain owners on data quality, system-of-record alignment, lineage, and feedback loops. Evaluation and quality engineering
? Create representative evaluation datasets and automated test suites for groundedness, relevance, correctness, task completion, refusal behaviour, safety, robustness, latency, and cost.
? Run regression testing across prompt, model, retrieval, tool, and policy changes; analyse failure modes and improve the system using trace-based evidence.
? Instrument online quality and business metrics, support controlled experiments, and incorporate human review for higher-risk or lower-confidence outcomes. Production operations and MLOps
? Build CI/CD pipelines for code, configuration, prompts, evaluation assets, and model or index changes across separated development, test, and production environments.
? Implement structured logging, tracing, token and infrastructure cost monitoring, model/provider health checks, alerting, dashboards, and operational runbooks.
? Optimize throughput, latency, reliability, and cost using caching, batching, routing, prompt/context management, and appropriately sized models.
? Participate in production support, incident response, root-cause analysis, and continuous improvement.
Security and responsible implementation:
? Implement controls for prompt injection, jailbreak attempts, unsafe tool use, data leakage, malicious content, model abuse, and dependency/supply-chain risk.
? Apply DIRECTV requirements for PII and payment-card data, identity and access, secrets management, retention, content rights, audit logging, and approved model/provider use.
? Contribute reusable components to the AI control plane, including policy enforcement, prompt/model configuration, evaluation hooks, telemetry, and kill-switch or rollback mechanisms.
? Work with Product Managers, UX, Solution Architects, AI Architects, Data Engineers, Cybersecurity, Quality Engineering, and Operations to deliver testable user outcomes.
? Write maintainable code, automated tests, interface contracts, technical documentation, deployment guides, and operational runbooks; participate in code and design reviews.
Required qualifications:
Typically, 10+ years in professional software engineering, including meaningful hands-on experience delivering AI, machine-learning, search, NLP, or data-intensive applications to production; equivalent experience is welcome.
Hands-on experience with LLM APIs, prompt and context design, RAG, embedding/search systems, structured outputs, tool/function calling, and automated evaluation.
Experience with SQL and document/search stores, containers, CI/CD, source control, cloud services, and observability practices.
Strong software engineering habits: modular design, automated testing, secure coding, peer review, performance troubleshooting, and production ownership.
Ability to explain model limitations and engineering trade-offs to technical and nontechnical partners.
Bachelor's degree in computer science, engineering, data science, or a related field, or equivalent practical experience.
#J-18808-Ljbffr