Job Description
We are looking for an experienced AI Conversational Engineer with strong expertise in real-time voice systems, Reinforcement Learning (RL)based model alignment, and custom LLM orchestration using PipeCat.
You will architect and optimize end-to-end conversational pipelines from Speech-to-Text and Text-to-Speech systems to SLM reinforcement and multi-agent orchestration ensuring low-latency, high-accuracy interactions at scale.
Core Engineering Responsibilities
1. Real-Time Conversational Systems
A. Design and build low-latency, high-concurrency conversational systems for both voice and text.
B. Integrate STT, TTS, and LLM/SLM components into unified, real-time architectures.
C. Develop and maintain PipeCat-based orchestration pipelines for multi-agent conversational flows.
D. Engineer robust streaming APIs and telephony integrations (VoIP/SIP).
2. Reinforcement Learning and Model Fine-Tuning
A. Fine-tune Small Language Models (SLMs) using Supervised Fine-Tuning (SFT) and RL (DPO, PPO) for alignment and personality control.
B. Design reward models to guide tone, factual accuracy, and conversational flow.
C. Build RL feedback loops for continuous model refinement based on user interactions.
3. Voice Synthesis and Adaptation
A. Develop high-quality ASR and TTS models for expressive, natural-sounding speech generation.
B. Apply speaker adaptation and voice cloning techniques for personalization.
C. Utilize Diffusion- or HiFi-GANbased vocoders for high-fidelity audio generation.
D. Engineer robust handling of sampling frequency, audio fidelity, and streaming performance.
4. Infrastructure, Serving, and Deployment
A. Build containerized inference microservices using Docker and Kubernetes.
B. Deploy Ray Servebased endpoints for distributed, dynamically batched inference.
C. Implement autoscaling, monitoring, and observability for production-grade systems.
D. Optimize serving for latency, throughput, and fault tolerance.
5. Guardrails, Security, and Reliability
A. Implement guardrail frameworks to protect against prompt injection, jailbreaks, and unsafe outputs.
B. Develop input sanitizers, content filters, and boundary-check mechanisms.
C. Maintain secure integrations with authenticated APIs and external toolchains.
D. Enable traceability through conversation logging, replay, and audit pipelines.
What We're Looking For:
3+ years in AI conversational systems or RL-driven model architectures.
Languages & Frameworks: Python, PyTorch, TensorFlow.
Core Expertise:
1) RL-based model alignment (SFT, PPO, DPO)
2) ASR/TTS pipeline design and optimization
3) Transformer architecture and optimization
4) Ray Serve + Kubernetes deployment
5) Secure orchestration using PipeCat
6) Programming & Engineering
Foundational Knowledge: Optimization, Statistics, and Linear Algebra.
Preferred Qualifications:
Bachelor's/Master's Degree in Computer Science or equivalent
Job Classification
Industry: Banking
Functional Area / Department: Data Science & Analytics
Role Category: Data Science & Machine Learning
Role: Data Scientist
Employement Type: Full time
Contact Details:
Company: Paytm
Location(s): Noida, Gurugram
Keyskills:
data scientist
kubernetes
python
tensorflow
data science
ai
pytorch
llm
microservices
docker
reinforcement learning
Data Science - Lead Data Scientist (Voice+Text)
