Lead Engineer (AI/ML - Training & Experimentation)
- Prodapt Solutions
- 5 - 10 years
- Chennai
- 3 months ago
- Email to a friend
- Report this job
Job Description
OverviewJoin the Prodapt team in building a unified, cloud-native environment for scalable machine learning training and experimentation. You will help design, develop, and optimize robust workflows that empower data scientists and engineers to efficiently explore, train, and validate ML models at scale.
ResponsibilitiesDevelop, maintain, and enhance interactive Jupyter-based notebook environments for model development, experimentation, and training.
Build and optimize Pipelines (file-based and git-based) for orchestrating training and experiment jobs, ensuring seamless transition from research to production.
Integrate and manage Project Namespace storage for organizing research data and experiment artifacts.
Enable flexible compute resource allocation (CPU/GPU) for diverse ML workloads, including deep learning, NLP, and recommendation systems.
Implement and support experiment tracking tools for managing model versions, metadata, and reproducibility.
Develop and optimize data loading and processing workflows using technologies such as Spark, cuDF, RAPIDS, and NVTabular. Leverage Google Cloud Platform (GCP) for scalable compute, storage, and orchestration of training jobs. Ensure robust model governance, including lineage tracking, metadata management, and compliance. Collaborate with data scientists, ML engineers, and platform teams to deliver reliable, production-grade solutions. Participate in code reviews, architecture discussions, and continuous improvement of the platform.Requirements
Required Technical Skills
- Proficiency in Python, especially for ML model development and data engineering.
- Experience with Jupyter Notebooks and interactive development environments.
- Familiarity with distributed data processing frameworks (Spark, cuDF, RAPIDS, NVTabular).
- Experience with cloud platforms (GCP preferred) and scalable compute resource management (CPU/GPU).
- Strong understanding of experiment tracking, model versioning, and metadata management.
- Experience with CI/CD tools and automation for ML workflows.
- Knowledge of model governance, reproducibility, and compliance best practices.
- Excellent troubleshooting, debugging, and communication skills.
Preferred Qualifications
- Experience with Pipeline orchestration (file-based and git-based).
- Familiarity with ecosystem or large-scale financial/ML platforms.
- Exposure to research and production environment isolation for ML workflows.
- Experience with dashboarding and monitoring tools for training jobs and experiment outcomes.
- Knowledge of data privacy, security, and compliance in ML environments.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data Platform EngineerEmployement Type: Full timeContact Details:
Company: Prodapt SolutionsLocation(s): Chennai
Keyskills: python data engineering model development debugging ml continuous integration natural language processing metadata management jupyter notebook aiml machine learning artificial intelligence sql deep learning spark gcp troubleshooting
Similar positions
Prodapt Solutions
Prodapt is a two-decade-old consulting & managed services provider, singularly focused on the telecom/DSP (digital service provider) ecosystem that helps clients transform their IT, products, operations, and networks to meet their strategic objectives. Prodapt provides end-to-end IT/software arc...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in