
Deep Rathi
AI Engineer | LLM Platforms & Distributed Systems
I build LLM systems that run in production: agent pipelines, retrieval, and the infrastructure under them.

About Me
Designing the future with AI and Code.
My Journey
I am the AI Lead at Key AI, where I build production LLM systems end to end: a Temporal-orchestrated multi-agent research pipeline, a member-facing assistant with 54 tools and tiered graph retrieval, a real-time conversational avatar for a Fortune 500 investment bank, and the discovery and recommendation surfaces behind the community product. Most of my work sits where model behaviour meets distributed systems: fan-out under concurrency, failure isolation, retrieval quality, and keeping latency and cost honest. I also contribute to open source, most recently the AdaBoost classifier now shipping in the Rust linfa-ensemble crate.
📌 Key Highlights
Tech Stack
Tools and technologies I work with
Languages & Core
Machine Learning & AI
LLM & Retrieval
Systems & Infrastructure
Web Development & DevOps
Specialized Domains
Experience
My professional journey
AI Lead
Aug 2025 – Present- •Scaled the multi-agent research pipeline from 11 to 200 concurrent runs by root-causing a per-token Kafka publish wedge; built it on Temporal with approval gates, fan-out across a 75-agent catalogue, mid-stream provider failover and per-agent failure isolation.
- •Shipped a real-time conversational avatar for a Fortune 500 investment bank at 0.9 s from end of speech to first audio: streaming STT, a fine-tuned LLM, streaming TTS, MuseTalk lip-sync and LivePortrait delivered over WebRTC.
- •Built Kai, the member-facing AI assistant, used by 23% of monthly active members: 54 tools behind a turn router, with tiered graph traversal across member, community and event entities and answers grounded on cited entity ids.
- •Designed the community site builder behind onboarding: a canonical document model that plans and writes 10–12 section sites, with roughly 70% of generated sections accepted without an edit.
- •Wired opportunity discovery over multi-vector Qdrant profiles with faceted filters, surfacing 3,400 of 22,000 monthly candidates, plus reciprocal intros that pair 6 bidirectional intents across shared communities.
Machine Learning Intern
Feb 2025 – Jul 2025- •Fine-tuned LLaMA 3 on 40,000 interview questions and shipped Gemini-backed USMLE and DSA question generators.
- •Wrapped SadTalker behind an API over a 13-persona library, returning an avatar video from a voice clip in under 5 s.
Data Scientist Intern
Aug 2023 – Oct 2023- •Developed interactive data dashboards (Tableau, Python, SQL) analyzing HR datasets for actionable business insights.
- •Collaborated with cross-functional teams to deliver data-driven solutions for operational improvements.
Research Intern
Jan 2022 – Oct 2022- •Contributed to LaneScan Net, a deep learning model for obstacle lane detection in autonomous driving.
- •Labelled over 10,000 driving images for the training set and supported model training and evaluation with VIT and SUNY Binghamton researchers.
Featured Projects
A showcase of my recent work in AI, Machine Learning, and Software Development.
rust-ml/linfa — AdaBoost
Wrote the AdaBoost (SAMME) classifier that now ships in the linfa-ensemble crate, version 0.8.1 on crates.io: the boosting loop with adaptive sample reweighting, per-model alpha weighting and weighted majority voting, behind a typed ParamGuard hyperparameter builder. Merged into rust-ml/linfa after maintainer review. Also fixed an ndarray version drift that was breaking CI across the workspace sub-crates.
PrecisionEdge
An LLM-driven data-analysis agent that takes a raw dataset and a question in plain language, then plans and runs the analysis: cleaning, feature preparation and generated insight. Built for the IIM Ahmedabad AI Hackathon 2024, where it took 1st place, and later written up and published in AIP Conference Proceedings.
DeribitTradingSystem
Low-latency C++ client for the Deribit derivatives exchange REST API: authentication, order placement, edit and cancel, open orders, positions and order book. The request path is built for latency — a pooled curl-handle transport, preallocated 16 KB response buffers, an order-book cache, TCP keepalive, a 10-minute DNS cache and branch-prediction hints on the hot path. Every trading and market-data request is timed with std::chrono, with round-trip latency aggregated in a fixed-capacity circular buffer reporting median and 95th percentile. A local Boost.Beast WebSocket server broadcasts order-book updates to subscribed clients.
GROW (AI Learning Platform)
Intelligent personalized learning platform powered by GROQ LLM API. Generates custom study plans in under 5 seconds based on user goals and learning style. Includes progress tracking, adaptive difficulty adjustment, and exportable study materials. Leverages fast inference for real-time educational content generation.
Sad-Talker-Custom
SadTalker wrapped behind a serving API over a 13-persona library, returning an avatar video from a voice clip in under 5 seconds. Adds persona selection, idle and blink handling and an inference entry point suited to being called from a product rather than a notebook.
DSA Question Generator
Algorithmic system that generates unlimited data structures and algorithm problems with automatic validation. Features difficulty scaling, constraint generation, and solution verification. Includes comprehensive test case generation and performance analysis. Useful for interview preparation, competitive programming, and algorithm education.
SHL Recommender
Collaborative filtering-based recommendation engine with advanced feature engineering and model optimization. Implements matrix factorization, neural collaborative filtering, and hybrid recommendation approaches. Optimized for both accuracy and computational efficiency with production-ready data pipelines.
Achievements & Publications
Recognition and contributions to the field
AI-driven ad generation using Kolmogorov-Arnold networks
AIP Conference Proceedings, vol. 3449 (2026). An AI-driven system for generating personalised multimedia advertisements from e-commerce interactions using Kolmogorov-Arnold networks. Fourth author.
View Paper→PrecisionEdge: Cutting-edge data insights
AIP Conference Proceedings, vol. 3449 (2026). An open application that automates data analysis using large language models. Second author.
View Paper→Winner, IIM Ahmedabad AI Hackathon 2024
Secured 1st place for developing an innovative AI solution under time constraints.
Get In Touch
Have a project in mind or just want to say hi?