Module Lead - Lead
- Happiest Minds
- 5 - 8 years
- Bengaluru
- 2 months ago
- Email to a friend
- Report this job
Job Description
Job Title: Senior Data Quality & Validation Engineer (AI-Integrated Data Systems)
Experience Level: 58 Years
Location: Bangalore
About the Role
We are seeking a highly skilled Senior Data Quality & Validation Engineer with strong expertise in data testing, orchestration, and AI-assisted development tools. The ideal candidate will bring hands-on experience validating batch and streaming data pipelines, implementing data quality frameworks, and leveraging AI productivity tools such as GitHub Copilot, Codex, or Cursor.
You will collaborate with cross-functional data, analytics, and AI teams to ensure the integrity, accuracy, and reliability of enterprise-grade data systems across cloud-based ecosystems (AWS, Azure, GCP).
Key Responsibilities
- Design, implement, and execute comprehensive test strategies for both batch and streaming data pipelines.
- Develop and maintain data validation and reconciliation frameworks to ensure accuracy and consistency across data layers.
- Perform complex SQL-based data validation including joins, window functions, CTEs, and performance optimization.
- Collaborate with data engineering teams to validate data models, perform synthetic data generation, and define data quality rulesets.
- Integrate and automate data quality checks using Python, Airflow DAGs, and orchestration frameworks.
- Work with BigQuery, Redshift, Synapse, or Snowflake for analytical data validation and performance tuning.
- Utilize Looker, Power BI, or equivalent BI tools for data consistency verification and visualization-based testing.
- Apply Prompt Engineering techniques to enhance test automation and data validation workflows.
- Implement and leverage MCP (Model Context Protocol) for AI-assisted data validation and integration with LLM-driven systems.
- Collaborate on AI-based data validation and testing tools leveraging Codex, GitHub Copilot, or Cursor.
- Optionally, experiment with OpenAI APIs and LLM configuration to improve automation in data quality processes.
- Support exploration and orchestration of AI agents for data pipeline monitoring and validation workflows.
Required Qualifications
- Bachelors or Masters degree in Computer Science, Data Engineering, or a related technical field.
- 58 years of hands-on experience testing and validating data pipelines (batch and streaming).
- Deep expertise with BigQuery, Redshift, Synapse, or equivalent data warehousing systems.
- Experience with Snowflake and Looker or similar analytics/BI platforms on AWS or Azure.
- Advanced SQL proficiency joins, CTEs, window functions, query optimization, and large-scale data validation.
- Strong command of Python for data validation, automation scripting, and framework integration.
- Working knowledge of Airflow or other data orchestration tools.
- Proven experience in data modeling, synthetic data creation, and defining data validation rule sets.
- Mandatory: Prompt Engineering, Codex/GitHub Copilot/Cursor usage, and MCP implementation & usage.
Preferred / Nice-to-Have Skills
- Familiarity with OpenAI APIs and LLM configuration.
- Understanding of LLM fundamentals and AI Agent orchestration.
- Experience integrating AI-driven automation within data engineering pipelines.
Soft Skills
- Strong analytical and problem-solving abilities.
- Excellent communication and collaboration skills across technical and non-technical teams.
- Self-driven, detail-oriented, and comfortable working in a fast-paced, innovation-driven environment.
Why Join Us
- Work at the intersection of Data Engineering and Generative AI.
- Contribute to next-generation AI-integrated data validation frameworks.
- Collaborate with a team passionate about innovation, automation, and data excellence.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Technical LeadEmployement Type: Full timeContact Details:
Company: Happiest MindsLocation(s): Bengaluru
Keyskills: cursor python github orchestration analytical data validation data testing automation testing amazon redshift usage data warehousing engineering integration framework artificial intelligence sql system collaboration bigquery mcp automation studio aws communication skills
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in