Azure Data Engineer
- Careernet
- 6 - 10 years
- Pune
- 6 hours ago
- Email to a friend
- Report this job
Job Description
Azure Data Engineer(SSE/DE/SDE)
Hyderabad, Chennai, Bangalore
JOB DESCRIPTION
Role Overview: SDE/DE
We are seeking technically strong Senior Data Engineers / Leads who can take ownership of designing and developing cutting-edge
data and AI platforms using big data and cloud technologies on Microsoft Azure.
You will play a critical role in building scalable data pipelines, modern data architectures, and intelligent analytics solutions.
Key Responsibilities:
Design and implement scalable, metadata-driven frameworks for data ingestion, quality, and transformation across both batch
and streaming datasets.
Develop and optimize end-to-end data pipelines to process structured and unstructured data, enabling the creation of analytical
data products.
Build robust exception handling, logging, and monitoring mechanisms for better observability and operational support.
Take ownership of complex modules and lead the development of critical data workflows and components.
Provide guidance to data engineers and peers on best practices, conduct code reviews, and enforces standards.
Collaborate with cross-functional teamsincluding business consultants, data architects, scientists, and application
developersto deliver impactful analytics solutions.
EXPERTISE AND QUALIFICATIONS
External Skills and Expertise:
510 years of total IT experience, including at least 3 years in big data engineering on Microsoft Azure.
Strong SQL expertise with experience in writing, optimizing, and troubleshooting complex queries on Azure SQL, Synapse, or
similar cloud databases.
Solid understanding of Spark architecture and core APIs - RDD, Dataframe, Dataset.
Strong understanding of the Databricks ecosystem, including Notebooks, Workflows, Unity Catalog, SQL Warehouse, Serverless
compute, latest databricks features.
Proven expertise in designing and developing scalable, high-performance data pipelines and automated workflows in Azure
Databricks leveraging PySpark, Spark SQL.
Experience in designing, building and orchestrating complex, parameterized pipelines in Azure Data Factory.
Familiarity with both batch and streaming data processing.
Solid understanding of data-modelling techniques (dimensional, 3NF) and data warehousing concepts
Experience delivering at least one end-to-end Data Lakehouse solution in Azure using the Medallion Architecture
Knowledge of different file formats such as Delta Lake, Avro, Parquet, JSON, and CSV
Advanced programming, unit-testing, and debugging skills in Python, PySpark, and SQL
Knowledge of data security and lifecycle management policies across Azure environment.
Familiarity with DevOps practices (CI/CD, Git, automated deployments).
Collaborative mindset with enthusiasm for working with stakeholders across the organization and taking ownership of
deliverables.
Good to Have:
Exposure to developing LLM/Generative AI-powered applications.
Knowledge about NoSQL database.
Experience in supporting BI and Data Science teams in consuming the data in a secure and governed manner.
Relevant Certifications on Microsoft Azure or Databricks are valuable addition.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: CareernetLocation(s): Pune
Keyskills: ADF PySpark Azure Databricks Medallion Architecture Data Engineer Spark SQL Big Data SQL Azure Data Factory Synapse Data Lakehouse Delta Lake ETL Python
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in