Adobe is seeking a highly skilled Senior Data Engineer to join our Doc Cloud Product Business Analytics organization. This role focuses on building and operating large-scale analytical data pipelines and analytics-ready data tables that power product adoption, usage metrics, experimentation (A/B testing), and executive dashboards across a massive monthly active user base.
You will work primarily on Azure Databricks, playing a key role in setting up and evolving our Databricks-based analytics data warehouse, including curated tables, metric foundations, and scalable data models.
This role is ideal for a hands-on data engineer with deep expertise in SQL, Python, Spark, and Delta Lake, and a strong understanding of analytics-driven data warehousing and product analytics.
Key Responsibilities
Design, build, and maintain large-scale, production-grade data pipelines on Azure Databricks using Apache Spark and Python
Write and optimize complex, high-performance SQL for data transformation, aggregation, and analytics workloads at scale
Develop and maintain analytics-ready data models (fact tables, dimensions, rollups, metric layers, and gold layer tables) used for dashboards and reporting
Optimize Databricks workloads for performance, reliability, and cost efficiency , including tuning Spark jobs and Delta Lake tables
Establish and apply Delta Lake best practices , including incremental and idempotent processing, MERGE patterns, partitioning, and table optimization.
Partner closely with product analysts, business analysts, data scientists, and product managers to enable reliable, self-serve analytics, supporting product-led growth use cases.
Implement data quality checks, validation frameworks, and monitoring to ensure accurate and trusted analytics metrics
Apply strong Data engineering best practices , including version control and documentation
Contribute to solutions that integrate structured and unstructured data , including selective use of GenAI / LLM-based capabilities where relevant
Required Qualifications
Education
bachelors degree or higher in Computer Science, Engineering, or a related field
Experience
6 12 years of professional experience in Data Engineering
Proven experience building and supporting production-grade analytical data pipelines at scale
Technical Expertise (Must-Have)
Expert-level SQL skills, including complex joins, window functions, performance tuning, and large-scale aggregations
Advanced Python proficiency for data processing, pipeline development, and automation
Deep hands-on experience with Azure Databricks and Apache Spark
Strong understanding of Delta Lake and optimization techniques (partitioning, Z-ordering, compaction)
Experience designing data models optimized for analytics and BI consumption
Cloud Platform
Strong experience with Microsoft Azure , including data lake storage and access controls
Familiarity with lakehouse architectures and enterprise data governance concepts
Preferred Qualifications
Experience with streaming or near real-time data pipelines on Databricks
Prior experience supporting product analytics, feature adoption, or MAU-based metrics
Exposure to MLOps, LLM deployment, or GenAI-enabled data applications
Familiarity with BI tools such as Tableau or Power BI and their performance considerations
Experience mentoring analysts or junior data engineers
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full time