Big Data Engineer
- Hucon Solutions India
- 1 Positions
- Bengaluru
- 8 months ago
- Email to a friend
- Report this job
Job Description
-
Design and implement large-scale data processing pipelines using Hadoop, Spark, and Scala.
-
Handle both batch and real-time data processing with high efficiency and fault tolerance.
-
Develop custom components and utilities for data ingestion and transformation.
-
Work with HDFS, YARN, and related Hadoop components for large-scale data storage and access.
-
Optimize Spark jobs for performance and reliability in a distributed environment.
-
Tune resource allocation, data partitioning, and memory usage to maximize throughput.
-
Integrate data from multiple internal and external sources (databases, APIs, files).
-
Automate workflows using Apache Airflow, Oozie, or similar orchestration tools.
-
Ensure high data quality, consistency, and availability across the pipeline.
-
Partner with analysts, data scientists, and platform teams to understand requirements and deliver data solutions.
-
Document data processes, transformation logic, and pipeline architecture.
-
Participate in design reviews and contribute to code and process improvements.
-
Bachelor's or Masters degree in Computer Science, Engineering, or related field.
-
6-10 years of hands-on experience in Big Data Engineering.
-
Proficient in Scala programming, and strong experience with Apache Spark (core, SQL, streaming).
-
Deep understanding of the Hadoop ecosystem: HDFS, YARN, Hive, Sqoop, etc.
-
Familiarity with data partitioning, performance tuning, and distributed processing concepts.
-
Strong experience working with large-scale datasets in production environments.
-
Experience with cloud-native big data platforms (e.g., AWS EMR, Azure HDInsight, GCP Dataproc).
-
Knowledge of Kafka, NoSQL databases (e.g., Cassandra, HBase).
-
Exposure to CI/CD, Docker/Kubernetes, and DevOps practices.
-
Relevant certifications (Cloudera, Hortonworks, Databricks) are a plus.
Job Title: Big Data Engineer Hadoop, Spark, Scala
Experience: 6 -10 years
Salary Range: 15 - 30 LPA
Locations: Chennai, Bangalore
Job Summary:
We are looking for a highly skilled Big Data Engineer with expertise in Hadoop, Apache Spark, and Scala to join our data engineering team. The ideal candidate will design and develop robust, scalable big data solutions for processing high-volume datasets and enabling data-driven insights across the organization.
Key Roles & Responsibilities (R&Rs):
1. Data Pipeline Development
2. Distributed Systems Engineering
3. Data Integration & Workflow Orchestration
4. Collaboration & Documentation
Required Skills & Qualifications:
Preferred Skills (Nice to Have):
Employement Category:
Employement Type: Full timeIndustry: OthersRole Category: General / Other SoftwareFunctional Area: Not SpecifiedRole/Responsibilies: Big Data EngineerContact Details:
Company: Hucon Solutions IndiaLocation(s): Bengaluru
Keyskills: apache spark hadoop big data scala big data engineer
Similar positions
Hucon Solutions India
Hucon Solutions India Pvt.Ltd.Hucon Solutions is an Integrated HR Service Provider for all Corporates all over India. We are backed by a good ERP and enough experience in HR and related activities. It has helped generate career opportunities for more than a million individuals in India. Hucon Solu...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in