Data Engineer, AWS, Hadoop
- Synechron
- 7 - 12 years
- Bengaluru
- 2 days ago
- Email to a friend
- Report this job
Job Description
Job Title: Data Engineer AWS + Hadoop
Location: Bangalore
Experience: 7+ Years
About the Role
Were looking for a seasoned Data Engineer with hands-on expertise in AWS data services and the Hadoop ecosystem. You will design, build, and optimize batch/streaming data pipelines, enable reliable data ingestion/processing, and support analytics, ML, and BI use cases at scale.
Key Responsibilities
- Design and implement scalable ETL/ELT pipelines for batch and streaming workloads.
- Build data ingestion frameworks using Kafka/Kinesis, and process data with Spark (PySpark/Scala).
- Develop and optimize data lakes and data warehouses on AWS (S3, Glue, EMR, Athena, Redshift).
- Manage and tune Hadoop ecosystem components (HDFS, Hive, Spark, Oozie/Airflow, Sqoop).
- Model data (star/snowflake), manage schemas, partitioning, and metadata; ensure data quality (DQ checks).
- Implement data governance, security, and access controls (IAM, Lake Formation, encryption, key management).
- Set up orchestrations and CI/CD for data jobs (Airflow/AWS Step Functions, Jenkins/GitHub Actions).
- Monitor pipelines and optimize cost, performance, and reliability (CloudWatch, logs, metrics).
- Collaborate with Analytics/ML/BI teams; provide high-quality curated datasets and APIs/Views.
- Document solutions, conduct code reviews, and enforce engineering best practices.
Required Skills & Qualifications
- 7+ years in Data Engineering with large-scale distributed data systems.
- Strong experience with AWS data stack: S3, Glue, EMR, Athena, Lambda, Redshift, IAM, CloudWatch.
- Hands-on with Hadoop ecosystem: HDFS, Hive, Spark (PySpark/Scala), Kafka, Oozie/Airflow.
- Expertise in SQL (complex queries, performance tuning) and data modeling.
- Practical knowledge of streaming (Kafka/Kinesis, Spark Streaming/Structured Streaming).
- Experience with Python or Scala for data pipelines; Shell scripting.
- Familiarity with Orchestration (Airflow/AWS Step Functions) and CI/CD for data jobs.
- Strong understanding of security & governance (encryption, PII handling, RBAC, Lake Formation).
- Proficient with version control (Git) and containers (Docker) for reproducible jobs.
- Excellent problem-solving, communication, and collaboration skills.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full timeContact Details:
Company: SynechronLocation(s): Bengaluru
Keyskills: Pyspark Data Engineering Hadoop Kafka AWS SCALA Python SQL
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in