Azure Data Engineer_Pune_Immediate
- Atyeti
- 5 - 10 years
- Pune
- 8 months ago
- Email to a friend
- Report this job
Job Description
We are looking for a skilled and experienced Data Engineer with experience in building scalable data solutions on the Microsoft Azure ecosystem. The ideal candidate must have strong hands-on experience with Azure Databricks, along with Spark, Python and SQL expertise. Familiarity with Data Lake, Data Warehouse concepts, and end-to-end data pipelines is essential.
Key Responsibilities:
- Lead the design end-to-end data engineering solutions across cloud platforms.
- Develop and oversee robust data pipelines and ETL workflows using Python and Apache Spark.
- Implement scalable Delta Lake solutions for structured and semi-structured data.
- Orchestrate complex workflows using Databricks Workflows or Azure Data Factory.
- Translate business rules, retention metadata, and data governance policies into reusable, modular, and scalable pipeline components.
- Ensure adherence to data privacy, security, and compliance standards (e.g., GDPR, HIPAA, etc.).
- Mentor and guide junior data engineers, fostering best practices in coding, testing, and deployment.
- Collaborate with cross-functional teams including , analysts, and business stakeholders to align data solutions with business goals.
- Drive performance optimization, cost-efficiency, and innovation in data engineering practices.
Required Skills & Qualifications:
- 5+ years of experience in data engineering
- Expert-level proficiency in Python, Apache Spark, and Delta Lake.
- Strong experience with Databricks Workflows and/or Azure Data Factory.
- Deep understanding of data governance, metadata management, and business rule integration.
- Proven track record in implementing data privacy, security, and regulatory compliance in insurance or financial domains.
- Strong leadership, communication, and stakeholder management skills.
- Experience with cloud platforms such as Azure.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Big Data EngineerEmployement Type: Full timeContact Details:
Company: AtyetiLocation(s): Pune
Keyskills: Azure Data Pipeline Databricks Python SQL Pyspark Datafactory
Similar positions
Atyeti
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 5...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in