Who We Are
Checksum is a high-growth startup revolutionizing software testing with AI-powered, self-healing QA automation. Checksum’s platform eliminates the need for manual test maintenance by automatically generating and adapting tests based on real user behavior. Engineering teams can now release code faster without sacrificing quality—and our traction proves it: we’ve tripled ARR in the last 9 months. We are backed by Leap Global Partners and were formed at super{set}, whose co-founders have over $1B in exits to Microsoft and Salesforce.
What You’ll Do
Design, build and operate AI agents that plan, act, observe and learn across browser, API and data tools
Integrate models with tools such as Playwright, internal APIs, vector search, document stores and webhooks
Create evaluation harnesses, golden datasets and automated regressions for reliability and safety
Optimize latency, accuracy and cost using caching, routing, function calling, tool selection and streaming I/O
Collaborate on product UX for agent feedback, interruptions, approvals and failure recovery
Own the full lifecycle from prototype to production, including CI, experiments and on call rotations
What We’re Looking For
BSc in computer science or equivalent experience
Ability to work independently, and solve problems based on clear goals
Strong Python or TypeScript skills
Experience building evals, datasets and automated tests for ML or LLM systems
Experience building agents that control browsers with Playwright or Selenium
Knowledge of CI/CD, containerization and cloud deployments
Exposure to retrieval augmented generation, planning and memory, multi agent orchestration and toolformer style designs
Experience with Anthropic, OpenAI, Google and open source models such as Llama, plus model selection and routing
Background in testing automation, QA engineering or developer tools
Benefits & Perks
A strategic role in shaping the growth trajectory of a cutting-edge AI startup
Competitive salary, equity, and benefits package
Lunch everyday you are in the office
Monthly company happy hours
Hybrid Role
This is a hybrid role, 4x a week, in the San Francisco Office.
About the Position
We’re looking for a builder who ships. You have hands on experience building with LLMs, shown through shipped features or serious prototypes. You have worked with prompts, function calling, tool use, structured outputs, evals and failure handling. You turn ambiguous problems into iterative plans, wire up tools and feedback loops, and measure what matters. You are comfortable moving between prompt design, Python code, and product tradeoffs. You enjoy pairing with engineers and PMs, mentoring teammates, and owning outcomes in a dynamic startup environment.