Data Engineer
- Optum
- 3 - 7 years
- Hyderabad
- 5 months ago
- Email to a friend
- Report this job
Job Description
Data Engineer Consultant APC383 - 27 (Individual Contributor)
Position Overview:
OHBI is seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in programming, data management, and cloud infrastructure, with a focus on designing and implementing efficient data solutions. This role requires a minimum of 3 to 6 years of experience with a deep understanding of Azure services and infrastructure, ETL/ELT solutions, Snowflake, and artificial intelligence technologies. The candidate should have some knowledge of AI tools and applications to enhance workflows, automate tasks, and extract insights from data.
Key Responsibilities:
- AI Integration and Development: Design, develop, and implement artificial intelligence systems, including the use of AI to create new software and AI-driven solutions. Apply AI-powered tools for data analysis, task automation, and decision-making.
- Azure Infrastructure Management: Own and maintain all aspects of Azure infrastructure, recommending modifications to enhance reliability, availability, and scalability.
- Security Management: Manage security aspects of Azure infrastructure, including network, firewall, private endpoints, encryption, PIM, and permissions management using Azure RBAC and Databricks roles.
- Technical Troubleshooting: Diagnose and troubleshoot technical issues in a timely manner, identifying root causes and providing effective solutions.
- Infrastructure as Code: Create and maintain Azure Infrastructure as Code using Terraform and GitHub Actions.
- CI/CD Pipelines: Configure and maintain CI/CD pipelines using GitHub Actions for various Azure services such as ADF, Databricks, Storage, and Key Vault.
- Programming Expertise: Utilize your expertise in programming languages such as Python to develop and maintain data engineering solutions.
- Real-Time Data Streaming: Use Kafka for real-time data streaming and integration, ensuring efficient data flow and processing.
- Data Management: Proficiency in Snowflake for data wrangling and management, optimizing data structures for analysis.
- DBT Utilization: Build and maintain data marts and views using DBT, ensuring data is structured for optimal analysis.
- ETL/ELT Solutions: Design ETL/ELT solutions using tools like Azure Data Factory and Azure Databricks, leveraging methodologies to acquire data from various structured or semi-structured source systems.
- Communication: Strong communication skills to explain technical solutions and issues clearly in technical and non-technical terms ensuring understanding at the Engineering Lead (Delivery Owner) and Key Stakeholders (Business leaderships)
Qualifications:
- Minimum of 3 to 6 years of experience in designing ETL/ELT solutions using tools like Azure Data Factory and Azure Databricks, Snowflake, Snowflake tasks and streams, Microsoft Data Fabric, Iceberg etc.
- Knowledgeable in programming languages such as Python.
- 1-2 years knowledge in LLMs, machine learning, data science, and programming languages.
- 1-2 years knowledge using AI tools for data analysis, automation, and insight extraction from large datasets
- Experience with Kafka for real-time data streaming and integration.
- Proficiency in Snowflake for data wrangling and management.
- Some level of understanding of dbt to build and maintain data marts and views.
- In-depth understanding of managing security aspects of Azure infrastructure.
- Experience in creating and maintaining Azure Infrastructure as Code using Terraform and GitHub Actions.
- Ability to configure, set up, and maintain GitHub for various code repositories.
- Experience in creating and configuring CI/CD pipelines using GitHub Actions for various Azure services.
- Strong problem-solving skills and ability to diagnose and troubleshoot technical issues.
Excellent communication skills
Job Classification
Industry: Software ProductFunctional Area / Department: Engineering - Software & QARole Category: DBA / Data warehousingRole: Database Developer / EngineerEmployement Type: Full timeContact Details:
Company: OptumLocation(s): Hyderabad
Keyskills: Snowflake ETL Data Bricks Github Aiml
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in