Smartsheet is building the next generation of AI-powered strategic planning and work execution agents through SmartAssist, our intelligent agent platform. You will work closely with Engineering and AI Platform teams, embedded in a team working in a critical area and building toward a mature Agent Development Lifecycle (ADLC). You will architect how we design, build, and operationalize AI-driven solutions that transform work execution at enterprise scale.
You Will:
Design and build production-grade LLM-powered agents and workflows within Smartsheet, including architecture decisions on system design, data pipelines, and deployment strategies.
Develop and optimize prompts, RAG pipelines, and agent reasoning patterns; build evaluation frameworks to measure accuracy, hallucination rates, and performance across model versions.
Implement and manage end-to-end AI systems on cloud infrastructure (AWS, GCP, or Azure), including monitoring, optimization, and incident response.
Collaborate with Engineering, Product, and cross-functional teams to translate business requirements into technical AI solutions that drive real impact.
Mentor and guide other engineers on applied ML best practices, modern AI development patterns, and production considerations.
Establish best practices and standards for AI development, model governance, and responsible AI within the organization.
Stay current with the rapidly evolving AI/LLM landscape and evaluate emerging tools, frameworks, and techniques for Smartsheet's needs.
Qualifications
8+ years of software engineering experience, with at least 2 years working directly with LLMs in production.
Deep, hands-on experience with prompt engineering and context engineering; understanding how model behavior changes with framing, structure, and input design.
Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis.
Experience building or extending LLM evaluation frameworks; designed scorers, worked with golden datasets, and thought carefully about what good looks like.
Fluency in agent system design; able to engage as a peer on architectural tradeoffs that affect quality.
Strong Python skills; comfortable working in data-heavy environments (Databricks, Delta tables, or equivalent).
Ability to communicate complex quality findings (written and verbal) to both technical and non-technical stakeholders.
Strong cross-functional judgment; knows when to escalate, when to resolve independently, and how to build credibility across teams.
A bias for clarity in ambiguous situations; brings structure and a clear point of view rather than waiting for consensus.
Legally eligible to work in the U.S. on an ongoing basis.
BS or MS in Computer Science, a related field, or equivalent industry experience.
Requirements
Experience with MLflow or similar experiment tracking platforms.
Familiarity with CI-integrated evaluation pipelines.
Experience with multi-agent orchestration frameworks.
Prior work in an Applied AI or LLMOps function within a product company.
Benefits
Employer subsidized medical/vision and dental coverage for full-time employees.
401k Match to help you save for your future (50% of your contribution up to the first 6% of your eligible pay).
Monthly stipend to support your work and productivity.
Flexible Time Away Program, plus Sick Time Off.
US employees are automatically covered under Smartsheet-sponsored life insurance, short-term, and long-term disability plans.
US employees receive 12 paid holidays per year.
Up to 24 weeks of Parental Leave.
Personal paid Volunteer Day to support our community.
Opportunities for professional growth and development including access to Udemy online courses.
Company Funded Perks, including a counseling membership, local retail discounts, and your own personal Smartsheet account.
Teleworking options from any registered location in the U.S. (role specific).
Competitive base salary range for roles that may be hired in different geographic areas.