Data Engineer
- Optum
- 3 - 7 years
- Hyderabad
- 5 months ago
- Email to a friend
- Report this job
Job Description
Data Engineer Consultant APC383 - 27 (Individual Contributor)
Position Overview:
OHBI is seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in programming, data management, and cloud infrastructure, with a focus on designing and implementing efficient data solutions. This role requires a minimum of 3 to 6 years of experience with a deep understanding of Azure services and infrastructure, ETL/ELT solutions, Snowflake, and artificial intelligence technologies. The candidate should have some knowledge of AI tools and applications to enhance workflows, automate tasks, and extract insights from data.
Key Responsibilities:
- AI Integration and Development: Design, develop, and implement artificial intelligence systems, including the use of AI to create new software and AI-driven solutions. Apply AI-powered tools for data analysis, task automation, and decision-making.
- Azure Infrastructure Management: Own and maintain all aspects of Azure infrastructure, recommending modifications to enhance reliability, availability, and scalability.
- Security Management: Manage security aspects of Azure infrastructure, including network, firewall, private endpoints, encryption, PIM, and permissions management using Azure RBAC and Databricks roles.
- Technical Troubleshooting: Diagnose and troubleshoot technical issues in a timely manner, identifying root causes and providing effective solutions.
- Infrastructure as Code: Create and maintain Azure Infrastructure as Code using Terraform and GitHub Actions.
- CI/CD Pipelines: Configure and maintain CI/CD pipelines using GitHub Actions for various Azure services such as ADF, Databricks, Storage, and Key Vault.
- Programming Expertise: Utilize your expertise in programming languages such as Python to develop and maintain data engineering solutions.
- Real-Time Data Streaming: Use Kafka for real-time data streaming and integration, ensuring efficient data flow and processing.
- Data Management: Proficiency in Snowflake for data wrangling and management, optimizing data structures for analysis.
- DBT Utilization: Build and maintain data marts and views using DBT, ensuring data is structured for optimal analysis.
- ETL/ELT Solutions: Design ETL/ELT solutions using tools like Azure Data Factory and Azure Databricks, leveraging methodologies to acquire data from various structured or semi-structured source systems.
- Communication: Strong communication skills to explain technical solutions and issues clearly in technical and non-technical terms ensuring understanding at the Engineering Lead (Delivery Owner) and Key Stakeholders (Business leaderships)
Qualifications:
- Minimum of 3 to 6 years of experience in designing ETL/ELT solutions using tools like Azure Data Factory and Azure Databricks, Snowflake, Snowflake tasks and streams, Microsoft Data Fabric, Iceberg etc.
- Knowledgeable in programming languages such as Python.
- 1-2 years knowledge in LLMs, machine learning, data science, and programming languages.
- 1-2 years knowledge using AI tools for data analysis, automation, and insight extraction from large datasets
- Experience with Kafka for real-time data streaming and integration.
- Proficiency in Snowflake for data wrangling and management.
- Some level of understanding of dbt to build and maintain data marts and views.
- In-depth understanding of managing security aspects of Azure infrastructure.
- Experience in creating and maintaining Azure Infrastructure as Code using Terraform and GitHub Actions.
- Ability to configure, set up, and maintain GitHub for various code repositories.
- Experience in creating and configuring CI/CD pipelines using GitHub Actions for various Azure services.
- Strong problem-solving skills and ability to diagnose and troubleshoot technical issues.
Excellent communication skills
Job Classification
Industry: Software ProductFunctional Area / Department: Engineering - Software & QARole Category: DBA / Data warehousingRole: Database Developer / EngineerEmployement Type: Full timeContact Details:
Company: OptumLocation(s): Hyderabad
Keyskills: Snowflake ETL Data Bricks Github Aiml
Similar positions
Optum
With the playing field for software development radically altered by distributed computing applications that span from cloud to mobile to Internet of Things (IoT), efficiency is at a premium. OptumSoft streamlines the software writing process through a platform that replaces the ad hoc approach many...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in