Data & AI Engineer - Microsoft Fabric, Experimentation Data & Agent Development

Overview:

Job Title: Data & AI Engineer - Microsoft Fabric, Experimentation Data & Agent Development

Location:

Bellevue, WA/Remote

Experience:

6-8 Years

Job Summary

We are seeking a highly skilled

Data & AI Engineer

with strong expertise in Microsoft Fabric, Azure AI Services, data engineering, and AI-powered analytics solutions. The ideal candidate will design and manage scalable experimentation data platforms, build curated Silver/Gold data layers, develop AI-powered data agents, and support enterprise analytics and experimentation workflows.

This role requires hands-on expertise in Microsoft Fabric, PySpark, clickstream analytics, Adobe Customer Journey Analytics (CJA), Azure AI Services, and modern data engineering best practices.

Required Skills

Strong expertise in:

Microsoft Fabric Data Engineering Data Modeling Big Data Platforms Data Pipeline Design

Hands-on experience with:

PySpark Azure AI Services AI/ML Integration Microsoft Fabric Data Agents Azure Data Factory (ADF)

Strong understanding of:

Clickstream data Customer Analytics Adobe Analytics Adobe Customer Journey Analytics (CJA) Experimentation data platforms

Experience with:

Data validation and reconciliation Performance tuning and monitoring Telemetry pipelines Data governance and semantic modeling

Strong analytical and problem-solving skills Ecommerce domain experience preferred Key Responsibilities

Experimentation Data Enablement

Design, build, and maintain curated Silver-layer datasets in Microsoft Fabric for experimentation analytics and reporting Develop reusable data products including standardized tables and views for dashboards, scorecards, and ad hoc analysis Ensure datasets are analysis-ready, cleaned, deduplicated, and semantically aligned Data Gap Analysis & Modeling

Conduct gap assessments across experimentation KPIs, telemetry systems, and Silver-layer datasets Identify data quality, latency, schema, and join-key issues and recommend remediation strategies Improve data models using:

Facts and dimensions Conformance rules Surrogate keys Grain optimization

Gold Layer & Stakeholder Management

Conduct workshops with BI, reporting, experimentation, and engineering teams Define KPI calculations, attribution logic, governance standards, and refresh SLAs Create source-to-target mappings, data dictionaries, validation rules, and technical specifications Pipeline Engineering

Build and maintain robust data pipelines using:

Microsoft Fabric Pipelines Azure Data Factory (ADF) Telemetry/1DS pipelines

Implement orchestration, incremental loads, monitoring, and error handling mechanisms Data Validation & Reconciliation

Validate and reconcile datasets with Adobe Customer Journey Analytics (CJA) Build automated validation routines for:

Missing data Duplicate detection Schema drift Metric anomaly detection

Coordinate issue resolution with telemetry and engineering teams AI Agent Development

Design and build AI-powered data agents using:

Fabric Data Agents Copilot Azure OpenAI

Enable use cases such as:

Automated scorecards Narrative summaries Self-service analytics Q&A Metric definition assistants Data quality monitoring assistants

Define AI agent architecture, RBAC security, grounding sources, and evaluation metrics Governance & Operational Excellence

Maintain documentation for:

Data transformations Pipeline dependencies Metric definitions Validation rules

Implement best practices for:

Naming conventions Semantic consistency Versioning Cost optimization

Support operational monitoring, troubleshooting, and continuous improvement initiatives Preferred Qualifications

Experience supporting experimentation and customer analytics platforms Strong knowledge of ecommerce and digital analytics ecosystems Exposure to AI-driven analytics and self-service data platforms Strong collaboration and stakeholder communication skills