Consultant - Lead Data Engineer
- Affine Analytics
- 5 - 8 years
- Bengaluru
- 8 months ago
- Email to a friend
- Report this job
Job Description
- Strong experience with Python, SQL, pySpark, AWS Glue. Good to have - Shell Scripting, Kafka
- Good knowledge of DevOps pipeline usage (Jenkins, Bitbucket, EKS, Lightspeed)
- Experience of AWS tools (AWS S3, EC2, Athena, Redshift, Glue, EMR, Lambda, RDS, Kinesis, DynamoDB, QuickSight etc.).
- Orchestration using Airflow
- Good to have - Streaming technologies and processing engines, Kinesis, Kafka, Pub/Sub and Spark Streaming
- Good debugging skills
- Should have strong hands-on design and engineering background in AWS, across a wide range of AWS services with the ability to demonstrate working on large engagements.
- Strong experience and implementation of Data lakes, Data warehousing, Data Lakehouse architectures.
- Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
- Monitor data systems performance and implement optimization strategies.
- Leverage data controls to maintain data privacy, security, compliance, and quality for allocated areas of ownership.
- Demonstrable knowledge of applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review).
- Experience in Insurance domain preferred.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: DBA / Data warehousingRole: Data warehouse Architect / ConsultantEmployement Type: Full timeContact Details:
Company: Affine AnalyticsLocation(s): Bengaluru+ View Contact
Keyskills: Python RDS Shell Scripting Kafka AWS Glue DynamoDB Lightspeed EMR EKS pySpark Redshift SQL Jenkins QuickSight Glue EC2 Kinesis AWS S3 Bitbucket Athena Lambda
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in