Lead Data Engineer
- Iris Software
- 8 - 13 years
- Noida, Gurugram
- 16 hours ago
- Email to a friend
- Report this job
Job Description
- 8+ years of experience in Data Engineering.
- 5+ years of hands-on experience with AWS data services.
- Strong proficiency in Python and/or Scala.
- Experience building ETL/ELT pipelines at scale.
- Strong SQL and data modeling expertise (OLAP, dimensional modeling, lakehouse).
Hands-on experience with :
- o Spark / PySpark
- o Airflow or similar orchestration tools
- o REST APIs and microservices
- Experience using GitHub Copilot or similar AI-assisted development tools in enterprise environments.
- Solid understanding of IAM, encryption, networking, and cloud security best practices.
Preferred Qualifications
- AWS Certifications (e.g., AWS Certified Solutions Architect, AWS Certified Data Analytics).
- Experience with streaming technologies (Kafka, Kinesis).
- Experience with containerization (Docker, Kubernetes).
- Knowledge of Delta Lake, Iceberg, or Hudi.
- Experience implementing data observability solutions.
Leadership & Soft Skills
- Proven ability to lead engineering teams and drive architectural decisions.
- Strong stakeholder communication skills.
- Ability to balance delivery velocity with engineering excellence.
- Experience working in Agile/Scrum environments.
Nice to Have
- Experience implementing cost governance and FinOps best practices on AWS.
- Exposure to AI/ML pipeline integration.
- Experience designing multi-account AWS architectures.
Mandatory Competencies
- Data Science and Machine Learning - Data Science and Machine Learning - Python
- Cloud - AWS - AWS S3, S3 glacier, AWS EBS
- Beh - Communication
- Programming Language - Scala - Scala
- Big Data - Big Data - Pyspark
- Big Data - Big Data - SPARK
- Programming Language - Python - Apache Airflow
- Data Science and Machine Learning - Data Science and Machine Learning - Apache Spark
- Database - Database Programming - SQL
- Database - Oracle - Data Modelling
- Artificial Intelligence - AI Tools - GitHub Copilot
- Middleware - API Middleware - Microservices
- Agile - Agile - SCRUM
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Iris SoftwareLocation(s): Noida, Gurugram
Keyskills: data engineer kubernetes scala artificial intelligence sql microservices docker data science iam spark apache spark big data etl s3 python github cloud security data analytics airflow ai machine learning kafka scrum agile aws
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in