Data Architect
- Virtusa
- 10 - 15 years
- Hyderabad
- 8 days ago
- Email to a friend
- Report this job
Job Description
- Design, develop, and maintain efficient and reliable data pipelines using Java, Scala, Apache Spark and Confluent Cloud (Kafka, KStreams, kSQLDB, Schema Registry)
- Leverage Apache Spark (Java/Scala) for large-scale data processing and transformation.
- Experience with building, maintaining and debugging applications and data pipelines using Confluent Cloud (Kafka, KStreams, kSQLDB, Schema Registry).
- Build and optimize data storage solutions using NoSQL databases such as ScyllaDB and/or Cassandra.
- Experienced with AWS services required for Data Engineering such as EMR, ServerlessEMR, AWS Glue, CodeCommit, EC2, S3 etc.
- Familiarity with workflow orchestration tools such as Airflow
- Experience with building and deploying applications using Docker or AWS ECS or AWS EKS
- Well versed with code management using tools like GitHub and CI/CD pipelines and deployment of data pipelines on AWS cloud.
- Implement and manage search and analytics capabilities using AWS OpenSearch and/or Elasticsearch.
- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver effective solutions.
- Monitor and troubleshoot data pipelines to ensure data quality and performance.
- Implement data governance and data quality best practices.
- Automate data ingestion, processing, and deployment processes.
- Stay up-to-date with the latest data engineering trends and technologies.
- Contribute to the design and architecture of our data platform on AWS.
Job Classification
Industry: BankingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data Platform EngineerEmployement Type: Full timeContact Details:
Company: VirtusaLocation(s): Hyderabad
Keyskills: Data Architect Airflow S3 Java Data Engineering Data Pipelines Architecture Github Scala Debugging Cassandra Kafka Apache Spark Nosql Data Quality Docker Elasticsearch Ec2 Nosql Databases Spark Aws Data Governance
Similar positions
Virtusa
Virtusa is a leading worldwide provider of information technology (IT) consulting and outsourcing services. We help accelerate business outcomes for Global 2000 businesses in banking and financial services, insurance, healthcare, telecommunications and media.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in