GCP Big Data Engineer Lead Consultant
- PTC
- 4 to 8 Yrs
- All India
- 3 months ago
- Email to a friend
- Report this job
Job Description
- Role Overview: As a Data Engineer in the Competency Center, your role will involve developing solutions for data collection, transformation, storage, and management to enable data-driven decision making and efficient data analysis. You will focus on the technical aspects of data processing, integration, and delivery to ensure data accuracy, reliability, and accessibility in a timely manner. Your responsibilities will also include addressing scalability, cost-effectiveness, security, and supportability of the solutions. You will work with various data toolsets and architectural concepts both on-premises and in cloud environments, such as data warehousing, data lakes, and data mesh. Your tasks will encompass extraction, ingestion, synchronization of structured and unstructured data, processing organization, orchestration, and performance optimization of data processing. Key Responsibilities: - Design, build, and deploy data pipelines in the cloud using Python, GCP, and BigQuery - Utilize object-oriented programming, Python, and SQL for data processing - Implement data quality checks and frameworks like Clouddq and Pydeequ - Work with GCP cloud-based infrastructure and systems - Develop proficiency in data modeling techniques and database optimization - Optimize query performance, indexing, and data retrieval efficiency - Design, configure, and manage databases for optimal performance and reliability - Integrate data from multiple sources using ETL and ELT techniques for analysis - Troubleshoot complex data-related issues and deliver new features and functionalities - Optimize data storage architectures and implement techniques like partitioning, compression, and indexing - Monitor system performance, identify anomalies, and conduct root cause analysis - Train and mentor junior data engineers to provide guidance and knowledge transfer Qualifications Required: - 4+ years of experience as a Data Engineer with proficiency in Python, GCP, and BigQuery - Strong knowledge of cloud computing platforms, especially Google Cloud - Experience with composer or Apache Airflow, knowledge of Dataplex is a plus - Familiarity with dq dimensions and data integration tools and techniques - Proficiency in SQL, Python, data modeling, and database management systems - Ability to optimize query performance, data retrieval, and system efficiency - Experience in troubleshooting data-related issues and optimizing data storage - Strong communication skills for direct client interaction *Note: No additional details about the company were provided in the job description.*,
Employement Category:
Employement Type: Full timeIndustry: IT Services & ConsultingRole Category: Not SpecifiedFunctional Area: Not SpecifiedRole/Responsibilies: GCP Big Data Engineer Lead ConsultantContact Details:
Company: Lingaro GroupLocation(s): All India
Keyskills: Python GCP SQL database optimization query optimization indexing performance tuning NoSQL ETL BigQuery ELT
Similar positions
PTC
Osprosys offers a stress free, professional environment with no heavy targets, focusing on work life balance and career growth, making it an ideal place for long term success.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in