Databricks Engineer
- Newbie Soft Solutions
- 10 - 17 years
- Pune
- 2 months ago
- Email to a friend
- Report this job
Job Description
:
Job Title: Senior Databricks Architect / Lead Databricks Engineer
Experience: 10+ Years (Minimum 4+ Years in Databricks)
Locations: Chennai | Bangalore | Pune | Hyderabad
Work Mode: Hybrid 3 Days from Office
Job Overview
We are looking for a highly experienced Databricks Architect with 10+ years of IT experience and at least 4 years of deep, hands-on Databricks expertise. The candidate will lead the design and delivery of scalable, secure, and cost-optimized data platforms on the Databricks Lakehouse ecosystem.
This role requires strong architectural ownership, technical leadership, and the ability to guide engineering teams.
Key Responsibilities
- Lead end-to-end Databricks architecture: ingestion, transformation, governance, analytics.
- Design scalable Lakehouse Architecture using Delta Tables, schema evolution, ACID transactions, time-travel, and interoperability.
- Integrate Databricks with enterprise ETL / orchestration tools.
- Build optimized pipelines with Spark (DataFrame API & Spark SQL) using Python, Scala, and SQL.
- Utilize Databricks-native features:
- Delta Live Tables (DLT)
- Unity Catalog
- Workflows
- Lakeflow & Lakehouse Federation
- Perform cluster configuration, performance tuning, cost optimization, security & monitoring.
- Build real-time streaming pipelines using Structured Streaming + Autoloader.
- Implement DevOps & automation: Terraform, GitHub Actions, Azure DevOps CI/CD.
- Create scalable data pipeline architectures using Databricks best practices.
- Strong expertise in data modeling (conceptual, logical, physical).
- Collaborate closely with CXOs, architects, and global stakeholders.
- Prepare solution proposals, architecture diagrams, technical documents, RFx responses, and presentations.
- Mentor and guide engineering teams to deliver high-quality data solutions.
Required Skills
- 10+ years of IT experience with 4+ years hands-on in Databricks
- Deep expertise in:
- Lakehouse architecture & Delta ecosystem
- Spark DataFrame API, Spark SQL
- Python, Scala, SQL
- Unity Catalog, DLT, Workflows
- Structured Streaming & Autoloader
- CI/CD (GitHub Actions, Azure DevOps) & Terraform
- Strong skills in cluster performance, cost governance, data security, and monitoring
- Excellent communication & stakeholder management
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Newbie Soft SolutionsLocation(s): Pune
Keyskills: Lakehouse Architecture Databricks Engineer Spark ETL Python SQL
Similar positions
Newbie Soft Solutions
PureSoftware delivers IT services and digital solutions across industries including telecom, healthcare, and financial services. The company specializes in software engineering, product development, and digital transformation. Its career portal reflects roles in DevOps, QA, cloud technologies, and e...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in