Databricks Expert/ Data Engineer (Databricks)
- Capgemini
- 4 - 6 years
- Hyderabad
- 7 months ago
- Email to a friend
- Report this job
Job Description
Job Title: Databricks Expert / Data Engineer (Databricks)
Location: Pan India
Job Type: Full-Time
About the Role:
We are seeking a highly skilled Databricks Expert to join our team. The ideal candidate will have deep expertise in Apache Spark, Databricks Lakehouse Platform, and cloud data platforms (Azure, AWS, or GCP). You will be responsible for designing, developing, and optimizing scalable data pipelines and analytics solutions.
Key Responsibilities:
- Design and implement scalable data pipelines using Databricks and Apache Spark.
- Develop and maintain ETL/ELT workflows for structured and unstructured data.
- Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions.
- Optimize performance of Spark jobs and manage cluster configurations.
- Implement data quality, governance, and security best practices.
- Work with Delta Lake, Unity Catalog, and MLflow for advanced analytics and machine learning workflows.
- Monitor and troubleshoot production data pipelines and jobs.
Required Skills & Qualifications:
- 4+ years of experience in data engineering or big data development.
- Strong hands-on experience with Databricks and Apache Spark (PySpark or Scala).
- Proficiency in SQL, Python, and/or Scala.
- Experience with Delta Lake, Databricks SQL, and notebooks.
- Familiarity with cloud platforms (Azure Data Lake, AWS S3, or GCP BigQuery).
- Knowledge of CI/CD pipelines, Git, and DevOps practices.
- Understanding of data modelling, data warehousing, and data Lakehouse architecture.
Preferred Qualifications:
- Databricks certifications (e.g., Databricks Certified Data Engineer Associate/Professional).
- Experience with machine learning workflows using MLflow.
- Familiarity with Unity Catalog and data governance frameworks.
- Experience in real-time data processing using Spark Structured Streaming or Kafka.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: CapgeminiLocation(s): Hyderabad
Keyskills: Databricks Spark Apache
Similar positions
Capgemini
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 5...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in