RESUME SUBMISSION REQUIREMENTS: We are seeking a highly skilled Artificial Intelligence Programmer to lead our AI development team. In addition to submitting your resume, candidates must provide a concise summary (no longer than one side of a page) highlighting their experience in leading teams focused on our key short term AI initiatives. These include document summarization with hypertext links to source documents, real-time phone call feedback systems, the creation of generative AI deliverables (e.g., legal demand letters), agentic workflows, and ad hoc queries to databases. Experience in writing comprehensive architecture documents, epics, and acceptance criteria is also required. This role demands a visionary leader with proven expertise in AI-driven projects and team management.
About SmartAdvocate: SmartAdvocate is the award-winning, enterprise-class legal case-management platform trusted by hundreds of law firms. We are launching a multi-year program to weave Large Models, real-time speech analytics, predictive insight, and autonomous agents into every step of a legal matter. We seek a hands-on AI Architect / AI Tech Lead / Director of AI who can lead a team of AI developers to turn ambitious ideas into secure, scalable, production systems.
What You’ll Do
Own the AI Architecture
– Design the end-to-end stack (RAG pipelines, vector stores, GPU inference clusters, event-driven micro-services, real-time audio services, and agent orchestration—all hardened for HIPAA, SOC 2 and privileged work-product).
Set Technical Direction
– Evaluate GPT-4 / Claude / Gemini vs. open-weights (Llama 3, Mistral, Claude 3 Opus, etc.). Build fine-tuning and RAG pipelines with LangChain, LlamaIndex, CrewAI, AutoGen, MetaGPT, and deploy via vLLM/TGI on-prem or VPC GPUs. Establish coding and DevOps standards for the AI team.
Lead/Director Delivery
– Break down a 12-month roadmap into iterative releases (document summarization with hypertext links to source documents, live phone call feedback, generative AI deliverables including legal demand letters, agentic workflows, ad hoc queries to database), writing architecture docs, epics and feature acceptance criteria.
Hands-on Prototyping & Code Reviews
– Build reference implementations in Python/C#, mentor engineers, and enforce best practices for prompt engineering, evaluation harnesses, and CI/CD.
Security & Compliance Champion
– Implement HIPAA-ready de-identification, encryption, audit logging, and model-governance controls; draft architecture for BAAs and SOC 2 evidence.
Cross-Functional Collaboration
– Work with the CTO, product managers, and legal SMEs to align AI capabilities with user value, timelines, and budget.
Performance & Cost Optimization
– Right-size GPU resources, tune model latency, and refine retrieval techniques to deliver sub-second answers where needed.
Talent Development
– Guide a small team of 4-6 ML and backend engineers, fostering a culture of experimentation and high-quality engineering.
Agentic & Real-Time Systems
– Lead development of agentic AI that suggests and (with approval) executes multi-step workflows, plus live call / Zoom analysis that delivers sub-second feedback, sentiment scores, and action-item extraction.
Domain Intelligence
– Encode legal workflows and medical chronology logic so the platform drafts settlement demand letters, discovery requests/responses, brief summaries, predictive case scores, and detects relationship-risk via sentiment.
Must-Have Qualifications
10+ yrs
enterprise software;
5+ yrs
ML/AI;
2+ yrs
production LLM or generative-AI deployments.
Strong written and verbal English communication skills
are essential, as this role involves collaborating with cross-functional U.S.-based teams, writing technical documentation, and participating in product planning meetings.
Proven track record architecting
self-hosted LLM systems
(Llama 2/3, Mistral, Claude, etc.) with
fine-tuning, Retrieval-Augmented
and vector databases ( Pinecone, Weaviate, Qdrant, Elasticsearch vector search ).
Hands-on expertise with
real-time speech-to-text
(OpenAI Whisper, Azure/GCP Speech, Deepgram, or similar), real-time NLP analytics, and WebRTC/Socket.io streaming.
Fluency in
LLM frameworks
(LangChain, LlamaIndex) and agent orchestration; comfortable implementing function-calling and workflow agents. Production experience building
agent-orchestrated
workflows using frameworks such as
CrewAI, AutoGen, or MetaGPT .
Deep
Python
(FastAPI, asyncio)
and C#/.NET
skills; comfortable reviewing PRs in both.
Familiarity with
legal terminology, litigation lifecycles, and HIPAA-compliant healthcare data handling .
Kubernetes, Docker, and
that automate workflows, enhance client engagement, and deliver measurable productivity gains.
Demonstrated leadership: mentoring engineers, running agile ceremonies, and communicating trade-offs to executives.
Nice-to-Have
Prior work in
legal-tech
or other regulated domains (finance, healthcare)
MS SQL Server tuning; ASP.NET MVC/Web API.
Experience with PACER, Westlaw, LexisNexis, or contract-analysis tools (Harvey, Spellbook).
Exposure to monitoring stacks ( Langfuse, MLflow, Prometheus/Grafana ) and differential privacy/federated learning.
CUDA, ONNX-Runtime, or Rust/Go for high-performance inference services.
Why SmartAdvocate?
High Impact + Autonomy
– Architect the AI backbone that will redefine legal case management.
Modern Stack, Real Budgets
– State-of-the-art GPUs, freedom to choose the best open-source and commercial tech.
Inclusive Culture
– Direct access to the CTO, rapid decision cycles, and a collaborative, growth mindset.
Competitive Package
– We celebrate diversity and are committed to an inclusive workplace.
Ready to Lead the Future of Legal AI?
Apply on Indeed with your résumé and a one-page case study describing a production AI system you architected—highlight its scale, security controls, and measurable business impact.
Job Type: Full-time
Pay: $150,000.00 - $200,000.00 per year
Benefits:
401(k)
401(k) matching
Dental insurance
Flexible spending account
Health insurance
Health savings account
Life insurance
Paid time off
Parental leave
Vision insurance
Application Question(s):
Are you able to work in our Melville, NY office at least 50% of your time?
This role requires strong English communication skills. Are you fluent in both spoken and written English?
Work Location: In person
Company DescriptionAward-Winning Legal Case Management Software Featuring Built-in Artificial Intelligence.