Delphi scales AI chatbots to 100M vectors using Pinecone database

Delphi, a San Francisco AI startup that creates personalized “Digital Minds” chatbots, has successfully scaled its platform using Pinecone’s managed vector database to handle over 100 million stored vectors across 12,000+ namespaces. The partnership enabled Delphi to overcome critical scaling challenges that were threatening its ability to maintain real-time conversational performance as creators uploaded increasing amounts of content to train their AI personas.

The scaling challenge: Delphi’s Digital Minds were drowning in data as creators uploaded podcasts, PDFs, and social media content to train their personalized chatbots.

• Open-source vector stores buckled under the company’s needs, with indexes ballooning in size and causing latency spikes during live events.
• Delphi’s engineering team was spending weeks tuning indexes and managing sharding logic instead of building product features.
• Each new upload of content added complexity to the underlying systems, making real-time responsiveness increasingly difficult.

How Pinecone solved it: The managed vector database provided enterprise-grade infrastructure with built-in privacy and performance optimization.

• Each Digital Mind now operates within its own Pinecone namespace, ensuring privacy compliance and improving search performance by narrowing the retrieval surface area.
• Retrievals consistently return in under 100 milliseconds at the 95th percentile, accounting for less than 30% of Delphi’s strict one-second end-to-end latency target.
• Creator data can be deleted with a single API call, streamlining compliance and data management.

The technical architecture: Delphi uses a retrieval-augmented generation (RAG) pipeline powered by Pinecone’s object-storage-first approach.

• Content is ingested, cleaned, and chunked before being embedded using models from OpenAI, Anthropic, or Delphi’s own stack.
• Pinecone dynamically loads vectors when needed and offloads idle ones, aligning with Digital Minds’ bursty usage patterns rather than constant activity.
• The system automatically tunes algorithms based on namespace size, with some Digital Minds storing thousands of vectors while others contain millions.

In plain English: Think of this like a smart filing system that only pulls out the documents you need when you need them, rather than keeping everything spread across your desk all the time. When someone asks their Digital Mind a question, the system quickly finds the most relevant information from that person’s uploaded content and uses it to generate a response that sounds like them.

Scale and performance metrics: The platform now handles significant concurrent usage without scaling incidents.

• Delphi sustains approximately 20 queries per second globally, supporting conversations across multiple time zones.
• The system manages over 100 million stored vectors across 12,000+ namespaces without hitting scaling bottlenecks.
• Performance remains consistent even during spikes triggered by live events or major content uploads.

Creator diversity: Digital Minds vary dramatically in scope and content volume depending on their creators’ archives.

• Some creators upload relatively small datasets from social media feeds, essays, or course materials amounting to tens of thousands of words.
• Others contribute massive archives, with one expert providing hundreds of gigabytes of scanned PDFs spanning decades of marketing knowledge.
• Pinecone’s serverless architecture accommodates this variance without requiring manual optimization.

Why RAG remains essential: Both companies push back against suggestions that expanding LLM context windows will make retrieval-augmented generation obsolete.

• “Even if we have billion-token context windows, RAG will still be important,” said Samuel Spelsberg, Delphi’s co-founder and CTO. “You always want to surface the most relevant information. Otherwise you’re wasting money, increasing latency, and distracting the model.”
• Jeffrey Zhu, VP of Product at Pinecone, emphasized that “dumping in everything you have is inefficient and can lead to worse outcomes. Organizing and narrowing context isn’t just cheaper—it improves accuracy.”

Future ambitions: Delphi plans to scale to millions of Digital Minds, requiring support for at least five million namespaces in a single index.

• The company is developing “interview mode,” where Digital Minds can ask questions of their creators to fill knowledge gaps and lower barriers to entry.
• Delphi has evolved from its 2023 focus on celebrity “clones” to positioning Digital Minds as enterprise-ready tools for scaling knowledge, teaching, and expertise.

What they’re saying: Industry leaders emphasize the infrastructure requirements for next-generation AI applications.

• “With Pinecone, we don’t have to think about whether it will work,” Spelsberg explained. “That frees our engineering team to focus on application performance and product features rather than semantic similarity infrastructure.”
• “Agentic applications like these can’t be built on infrastructure that cracks under scale,” Zhu noted, highlighting Pinecone’s design for bursty, multi-tenant workloads.