How 24,000 companies keep their AI from Breaking in Production | Rohit Agarwal, Portkey
The Neon Show
- Published
- March 19, 2026
- Duration
- 1h 18m
- Summary source
- description
- Last updated
- Aug 23, 2026
Discusses openai, anthropic, google-ai.
Summary
Over 1 Trillion AI tokens pass through Portkey every single day.Every AI product eventually runs into the same problem. The prototype works, but once it goes live the system has to manage multiple models, rising token costs, unpredictable latency, and infrastructure that was never built for AI workloads.That is the problem Rohit Agarwal is solving with Po…
Intelligent Report
Sign in to read teasers, or upgrade to Research Pro to commission intelligent report for this episode. Learn more →
Show notes
Over 1 Trillion AI tokens pass through Portkey every single day.Every AI product eventually runs into the same problem. The prototype works, but once it goes live the system has to manage multiple models, rising token costs, unpredictable latency, and infrastructure that was never built for AI workloads.That is the problem Rohit Agarwal is solving with Portkey, an AI gateway that sits between applications and the models, whether that’s GPT-4, Claude, or Gemini.With 24,000 companies routing their
Themes
- openai
- anthropic
- google-ai