“Reranking is a crucial step of retrieval, and zerank-1 was the first reranker we tried that was actually accurate, but also fast and calibrated.”
— Deshraj Yadav, CTO at Mem0
Company and Highlights
Mem0 is the universal memory layer trusted by 50,000+ developers to power AI agents across healthcare, enterprise, education, and more. Memory is the key intelligence layer that lets AI recall facts, learn over time, and deliver personalization.
Mem0 migrated their production rerank traffic to ZeroEntropy’s zerank-1, a critical component of their retrieval stack. With ZeroEntropy, they get more calibrated scores, consistent latency distributions, and throughput at scale, now processing over 1B tokens per day with predictable performance.
- Throughput: around 1B tokens per day
- Production latency: p50 75 ms, p90 125 ms, p99 238 ms
- Predictable scaling across candidate set sizes
- Simple API swap for integration
- SOC 2 and HIPAA compliance
Problem
Mem0 powers AI Agents across industries, at scale. These agents rely on Mem0’s memory layer to surface the right facts in real time. As usage scaled, Mem0’s previous reranker became a bottleneck. Two problems kept surfacing:
- Noisy retrievals across verticals. Inconsistent scoring made it difficult to set thresholds that worked equally well for healthcare assistants, enterprise copilots, and consumer chatbots. What looked relevant in one domain often failed in another.
- Unpredictable latency. At high load, tail latencies spiked, breaking the seamless, real-time experience users expect from AI agents.
Approach
Mem0 tested ZeroEntropy’s zerank-1 reranker in a sandbox environment locally, thanks to the open-weights available on HuggingFace:
Benchmark
Benchmarked against internal metrics of accuracy, and calibration stability.
Evaluate
Evaluated impact on downstream customer use cases (retrieval accuracy, personalization fidelity, token savings).
Integrate
After confirming superior accuracy metrics, Mem0 started integrated ZeroEntropy’s API for production scale. Migration required a single API swap within Mem0’s retrieval-and-memory compression pipeline.
Results
Latency in Production
Calibration & Accuracy
- Scores were stable across domains, making thresholding simpler and improving retrieval consistency.
- Higher relevance fidelity translated into stronger personalization and context recall.
Scale
- Mem0 now processes over 1B tokens per day through ZeroEntropy rerankers with consistent SLO adherence.
- Predictable O(N) scaling allows Mem0 to increase candidate sets without breaching latency budgets.
Decision
Mem0 migrated production rerank traffic to ZeroEntropy, making our reranker a critical part of the memory infrastructure trusted by 50,000+ developers and enterprises worldwide.
“ZeroEntropy made it possible for us to deliver deliver high accuracy retrieval for our memory retrieval pipeline at scale”.
— Deshraj Yadav, CTO at Mem0
Why ZeroEntropy
- Latency & Tail Control: Stable p50–p99 latencies even at high throughput.
- Calibration: calibrated performance holds across diverse domains and workloads.
- Cost Efficiency: Token-based pricing aligned with Mem0’s usage model.
- Drop-In Integration: Minimal engineering lift for production rollout.
