Azure Data Engineer @ Infogain

Home > Software Development

Azure Data Engineer

Infogain
10 - 14 years
Noida, Gurugram
6 months ago
Email to a friend
Report this job

Job Description

Lead design and execution of Dataproc Databricks PySpark migration roadmap.
Define modernization strategy , including data ingestion, transformation, orchestration, and governance.
Architect scalable Delta Lake and Unity Catalog -based solutions.
Manage and guide teams on code conversion, dependency mapping, and data validation.
Collaborate with platform, infra, and DevOps teams to optimize compute costs and performance.
Own the automation & GenAI acceleration layer , integrating code parsers, lineage tools, and validation utilities.
Conduct performance benchmarking, cost optimization, and platform tuning (Photon, Auto-scaling, Delta Caching).
Mentor senior and mid-level developers, ensuring quality standards, documentation, and delivery timelines.

Technical Skills

Languages: Python, PySpark, SQL
Platforms: Databricks (Jobs, Workflows, Delta Live Tables, Unity Catalog), GCP Dataproc
Data Tools: Hadoop, Hive, Pig, Spark (RDD & DataFrame APIs), Delta Lake
Cloud & Integration: GCS, BigQuery, Pub/Sub, Cloud Composer, Airflow
Automation: GenAI-powered migration tools, custom Python utilities for code conversion
Version Control & DevOps: Git, Terraform, Jenkins, CI/CD pipelines
Other: Performance tuning, cost optimization, and lineage tracking with Unity Catalog

Preferred Experience

10-14 years of data engineering experience with at least 3 years leading Databricks or Spark modernization programs.
Proven success in migration or replatforming projects from Hadoop or Dataproc to Databricks.
Exposure to AI/GenAI in code transformation or data engineering automation .
Strong stakeholder management and technical leadership skills.

EXPERIENCE

11-12 Years

Job Classification

Industry: IT Services & Consulting
Functional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time

Contact Details:

Company: Infogain
Location(s): Noida, Gurugram

+ View Contact

Login

Candidates can login here to view contacts and apply.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach Resume Max 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Candidates are expected to provide most recent and accurate profile information, inappropriate content is strictly prohibited!

Keyskills: hive python technical leadership data validation performance tuning airflow pyspark apache pig data engineering artificial intelligence sql dataproc data bricks automation apache git stakeholder management spark gcp data ingestion hadoop bigquery

Job seems aged, it may have been expired!
Fraud Alert to job seekers!

₹ Not Disclosed

Job application

We will notify the employer with your details. You can also attach a resume or a cover letter.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach ResumeMax 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Similar positions

Devops Engineer

Fiserv

8 - 13 years

Pune

3 days ago

₹ Not Disclosed

Manager Site Reliability Engineer

Global Technology

8 - 12 years

Pune

14 hours ago

₹ 0-40 Lacs P.A.

Walk in - Devops Engineer (Terraform) - Hyderabad

Tata Consultancy

4 - 9 years

Hyderabad

15 hours ago

₹ Not Disclosed

Windows C++ Engineer

Quest Global

3 - 6 years

Pune

2 days ago

₹ Not Disclosed

Infogain

A Silicon-Valley headquartered company, Infogain is a global business oriented IT consulting provider of front-end, customer-facing technologies, processes and applications, leading to a more efficient and streamlined customer experience. We want our clientsâ‚¬â„¢ interactions with their cus...

Azure Data Engineer @ Infogain

Home > Software Development