Senior Data Engineer / AI Engineer (Agentic AI Platform – Financial Data)
Location: Philadelphia, PA – Hybrid (On-site 2-3 days per week)
Duration: Contract
Interview Mode: Video
About the Role
We are building a platform that converts unstructured financial data (emails, corporate actions, index announcements) into high-quality, structured datasets used by financial institutions.
This is not a typical "LLM wrapper" role. You will work on systems that:
Extract data from noisy, inconsistent sources
Validate and reconcile outputs across multiple inputs
Ensure correctness, traceability, and auditability
The challenge is not just applying LLMs—it's making them reliable in production for financial workflows.
What You'll Work On
Designing pipelines that process high-volume financial documents (batch + near real-time)
Building LLM-powered extraction workflows (classification, parsing, summarization)
Implementing validation layers (rule-based + model-based) to reduce hallucinations
Developing retrieval systems using embeddings and vector search
Architecting end-to-end systems: ingestion → processing → storage → serving
Ensuring data quality, observability, and fault tolerance
Collaborating with product to turn messy data into usable financial intelligence
Core Requirements (Must-Have)
Strong Python and backend/data engineering experience
Experience building production data pipelines (ETL, streaming, or async systems)
Solid understanding of distributed systems and failure modes
Experience working with LLM-based systems in production including:
Prompt design
Output validation
Retry/fallback strategies
Evaluation and monitoring
Experience with data storage systems (SQL + NoSQL)
Familiarity with cloud infrastructure (AWS or similar)
Preferred Experience
Experience with RAG / vector search systems
Background in financial data or capital markets
Experience with streaming systems (Kafka, etc.)
Experience building multi-step or agent-style workflows
What Makes This Role Interesting
Work on high-accuracy AI systems where correctness matters
Solve real problems around:
LLM reliability and hallucination mitigation
Data consistency across conflicting sources
Real-time vs correctness tradeoffs
Build systems used in financial decision-making workflows
High ownership over core architecture in an early-stage environment
Nice to Know (but not required)
Experience with orchestration tools (Airflow, etc.)
Exposure to evaluation frameworks for LLMs
Experience working with large-scale document processing
Tech Stack (Representative, not exhaustive)
Python, APIs, async processing
LLM APIs + embeddings
SQL / NoSQL databases
Cloud infrastructure (AWS)
Data pipelines and streaming systems
Vector Databases
Note for candidates: Local candidates only to Philadelphia, PA (no relocation accepted).