I
Senior AI Engineer
Insight Global
2 hours ago
Full-time
On-site
East New York, New York, United States
Required Skills & Experience
• Expert Python: 5+ years of production Python experience, including services the candidate personally operated; experience with async development, typing, Pydantic, packaging, and profiling.\n• Production AI: Hands-on experience with both production document extraction and retrieval at scale.\n• AI Evaluation: Personally built an evaluation framework/harness for a non-deterministic system and used the results to make a meaningful production decision.\n• Measurement & Statistics: Strong understanding of hypothesis testing, confidence intervals, sample sizing, precision/recall/F1, accuracy, variance, and model calibration.\n• Dataset & Error Analysis: Experience with sampling, annotation, dataset versioning, label quality/agreement, and identifying performance issues hidden within specific data populations.\n• Production Engineering: Experience with event-driven architectures, idempotency, deduplication, replay/backfill, schema/API evolution, and safe rollout/rollback practices.\n• Regulated Data: Experience appropriately handling sensitive/personal data, including classification, minimization, redaction, retention, least-privilege access, and secrets management.
Nice to Have Skills & Experience • Agentic AI / AI orchestration experience in production.\n• Experience managing LLM cost, token usage, and caching.\n• Online experimentation such as A/B testing, shadow deployments, or similar comparison approaches.\n• Drift detection and traditional ML experience.\n• Fine-tuning, adapter training, distillation, or systematic prompt optimization.\n• Large-scale analytical/data experience.\n• Experience with FERPA, HIPAA, GDPR, PCI-DSS, or SOX environments. \n• GCP and Google ADK experience would be a strong plus.\n• ML experience is beneficial.\n• Higher education experience is a plus, but not required.
Job Description
Position Overview:\nWe are seeking a Senior AI Engineer to build and support production AI solutions involving document intelligence, LLMs, retrieval, evaluation, and agentic AI. This is a hands-on engineering role for someone who has built, operated, measured, and improved AI systems in production, not someone whose experience is primarily research, notebooks, or proof-of-concepts. \n\nThe client operates a live document-intelligence platform: multimodal large language models extract structured records from high volumes of unstructured documents, hybrid retrieval reconciles those records against large reference catalogs, and human reviewers adjudicate low-confidence output through internal applications.\n\nWe are seeking one senior individual contributor to own the applied-AI and data-science layer end to end — extraction modeling, retrieval quality, and the evaluation and experimentation infrastructure that governs both.\n\n\nKey Responsibilities:\n• Build and operate production AI solutions for document extraction and information retrieval.\n• Develop reliable Python services supporting LLM and AI applications.\n• Build evaluation and experimentation frameworks to measure AI quality and guide production decisions.\n• Perform structured dataset analysis, error analysis, and model comparison.\n• Develop retrieval capabilities using dense/sparse retrieval, vector search, ranking, and reranking.\n• Support agentic AI and AI orchestration initiatives.\n• Build reliable event-driven services and safely manage application/schema changes.\n• Ensure sensitive and regulated data is appropriately protected across AI systems. \n\nWe are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here.
Nice to Have Skills & Experience • Agentic AI / AI orchestration experience in production.\n• Experience managing LLM cost, token usage, and caching.\n• Online experimentation such as A/B testing, shadow deployments, or similar comparison approaches.\n• Drift detection and traditional ML experience.\n• Fine-tuning, adapter training, distillation, or systematic prompt optimization.\n• Large-scale analytical/data experience.\n• Experience with FERPA, HIPAA, GDPR, PCI-DSS, or SOX environments. \n• GCP and Google ADK experience would be a strong plus.\n• ML experience is beneficial.\n• Higher education experience is a plus, but not required.
Job Description
Position Overview:\nWe are seeking a Senior AI Engineer to build and support production AI solutions involving document intelligence, LLMs, retrieval, evaluation, and agentic AI. This is a hands-on engineering role for someone who has built, operated, measured, and improved AI systems in production, not someone whose experience is primarily research, notebooks, or proof-of-concepts. \n\nThe client operates a live document-intelligence platform: multimodal large language models extract structured records from high volumes of unstructured documents, hybrid retrieval reconciles those records against large reference catalogs, and human reviewers adjudicate low-confidence output through internal applications.\n\nWe are seeking one senior individual contributor to own the applied-AI and data-science layer end to end — extraction modeling, retrieval quality, and the evaluation and experimentation infrastructure that governs both.\n\n\nKey Responsibilities:\n• Build and operate production AI solutions for document extraction and information retrieval.\n• Develop reliable Python services supporting LLM and AI applications.\n• Build evaluation and experimentation frameworks to measure AI quality and guide production decisions.\n• Perform structured dataset analysis, error analysis, and model comparison.\n• Develop retrieval capabilities using dense/sparse retrieval, vector search, ranking, and reranking.\n• Support agentic AI and AI orchestration initiatives.\n• Build reliable event-driven services and safely manage application/schema changes.\n• Ensure sensitive and regulated data is appropriately protected across AI systems. \n\nWe are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here.