Senior Python PySpark Data Engineer
- Infobeans
- 8 - 12 years
- Indore
- 2 months ago
- Email to a friend
- Report this job
Job Description
- Design, develop, and maintain scalable data pipelines and applications using Python and PySpark.
- Build end-to-end data solutions from scratch, ensuring clean, modular, and reusable code.
- Write and optimize complex SQL queries for large-scale data processing, transformation, and validation.
- Develop dynamic, reusable frameworks and implement design patterns for scalability and performance.
- Refactor legacy codebases to enhance performance, maintainability, and reliability.
- Collaborate with cross-functional teams including data engineers, analysts, and stakeholders to translate business requirements into technical solutions.
- Utilize Databricks and Azure Data Factory (ADF) for data workflow orchestration, automation, and monitoring.
- Drive best practices in code reviews, performance tuning, and deployment automation.
- Stay current with emerging tools and technologies in data engineering and contribute to continuous improvement initiatives.
- Youll have the freedom to build from the ground upcreating scalable, efficient, and production-grade data systems.
- Youll work with modern cloud technologies such as Databricks, ADF, and Azure.
- Youll collaborate with a team that values innovation, problem-solving, and clean engineering principles.
- Youll be part of a fast-paced, learning-driven environment where your ideas are heard and implemented.
- Youll get opportunities to influence architecture and design decisions for enterprise-level solutions.
- 8+ years of hands-on experience in software or data engineering with strong proficiency in Python and PySpark.
- Proven experience building Python applications from scratch with modular, well-documented code.
- Strong command over SQL and working with large datasets.
- Expertise in Databricks and Azure Data Factory (ADF).
- Solid understanding of data structures, design patterns, and scalable architectures.
- Experience in performance optimization, debugging, and automation.
- Exposure to cloud platforms such as Azure, AWS, or GCP.
- A problem-solver mindset with the ability to think innovatively and deliver end-to-end solutions.
- A Bachelors degree in Computer Science, Engineering, or related field.
Location - Indore,Pune,Nagar,Chennai,Bengaluru
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: InfobeansLocation(s): Indore
Keyskills: pyspark azure data factory sql python data bricks hive scala gcp spark design patterns oracle adf debugging data structures hadoop big data etl oracle sql queries performance tuning data processing airflow microsoft azure data engineering sqoop aws
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in