Full Stack Data Engineer
- Niveus Solution
- 5 - 8 years
- 2 months ago
- Email to a friend
- Report this job
Job Description
Key Responsibilities
Design, build, and maintain end-to-end GCP data pipelines (batch and
streaming).
Ensure data platform uptime and performance in alignment with defined
SLAs/SLOs.
Develop and optimize ETL/ELT workflows using Cloud Composer (Airflow),
Dataflow (Apache Beam), and BigQuery.
Manage and enhance data lakes and warehouses using BigQuery and Cloud
Storage.
Implement streaming data solutions using Pub/Sub, Dataflow, or Kafka.
Build data APIs and microservices for data consumption using Cloud Run,
Cloud Functions, or App Engine.
Define and enforce data quality, governance, and lineage using Data Catalog
and Cloud Data Quality tools.
Collaborate with DevOps to build CI/CD pipelines, infrastructure as code, and
automated monitoring for data workflows.
Participate in incident management, RCA, and change control processes
following ITIL best practices.
Proactively identify optimization opportunities to improve cost efficiency,
latency, and reliability.
Mentor junior engineers and ensure adherence to engineering best practices.
Technical Skills & Experience
Programming: Python (advanced), SQL (expert), Shell scripting.
GCP Data Stack:
o Data Storage: BigQuery, Cloud Storage
o Data Processing: Dataflow (Apache Beam), Dataproc (Spark), Cloud
Composer (Airflow)
o Streaming: Pub/Sub, Kafka (optional)
o Orchestration: Cloud Composer / Airflow
o APIs & Services: Cloud Run, Cloud Functions, API Gateway
o Monitoring: Cloud Logging, Cloud Monitoring, Stackdriver, or
Prometheus/Grafana
DevOps: Terraform, Git, Docker, Kubernetes, Cloud Build / Jenkins.
Data Visualization: Looker Studio, Power BI, or Tableau.
Version Control & CI/CD: GitHub, GitLab, or Bitbucket pipelines
Key Competencies
Proven experience in managed service delivery under strict SLAs (availability,
latency, and resolution timelines).
Strong understanding of GCP networking, IAM, and cost management.
Strong knowledge of incident management, change management, and
problem management using ITSM tools (e.g., ServiceNow, Jira).
Ability to lead on-call operations in a 16/5 support model, coordinating across
global teams.
Understanding of data security, governance, and compliance.
Excellent communication and client-handling skills.
Ability to work in a 16/5 model, coordinating with global teams across time
zones.
Good to Have
Certifications:
o Google Professional Data Engineer
o Google Professional Cloud Architect
Experience with data observability tools (Monte Carlo, Databand, or similar).
Exposure to MLOps pipelines on GCP (Vertex AI, AI Platform).
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data Platform EngineerEmployement Type: Full timeContact Details:
Company: Niveus SolutionLocation(s): Mangaluru
Keyskills: data networking problem management change management artificial intelligence sql apache iam gcp spark looker client cost management shell scripting api bigquery programming communication skills jira python dataproc incident management kafka cloud storage handling data flow
Similar positions
Niveus Solution
About us : With over 10+ years of proven expertise in technical consultation and related services, Niveus Solutions is a renowned name in providing technology and business solutions to modernize organizations using Google Cloud. As a Google Cloud Partner, Niveus has evolved as an outstanding ser...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in