Ex Alibaba engineer's new startup Octen pairs 10 millisecond AI search with a $20 unlimited plan
Edmund and Geffen dig into Octen’s “10‑millisecond, $20 unlimited” AI search pitch: what generative retrieval actually does, where the pricing and speed numbers are real, and when this changes how you’d design agents versus just being nicer infra marketing.
Transcript
Edmund So you’ve got Octen out here saying ten‑millisecond AI search and twenty bucks for unlimited queries… where does this stop being a fun tweet and start being real infra, Geffen?
Geffen Right now it mostly sounds like fun tweet to me. The headline is ten milliseconds, but that’s just the server‑side median for their Pulse model generating an ID for a URL. It’s not the whole journey from your agent to a final answer on screen.
Edmund Yeah.
Geffen You still pay for network hops, fetching the page, maybe pulling passages, then the L L M thinking over it. So I’m wary of people hearing “ten milliseconds” and imagining ChatGPT suddenly answering in the blink of an eye.
Edmund Sure, but they’re not really selling vibes, they’re selling a constraint shift. Kuan Zou’s whole argument in the piece is: agents can fan out over lots of data points at once, but today you ration external calls because search is slow and pricey. If they make each lookup cheap enough and fast enough, you stop architecting around “do I dare make another call?”
Geffen That’s the best version of it, yeah. And technically the Pulse thing is interesting. They’re doing generative retrieval: instead of a big inverted index plus reranking, the model just spits out an identifier that maps to a URL from a table of about two hundred one million pages. They clamp the output to that table so the model can’t hallucinate some fake domain.
Edmund So it’s like the model is generating a pointer, not prose.
Geffen Exactly. And on their own navigational traffic—so think “take me to the Stripe status page” type stuff—they claim about fifty‑nine point six percent accuracy on getting the right page first, versus forty‑six and a half for a dense retrieval baseline. That’s a decent jump, but it also quietly says forty percent of the time you still don’t land on the exact page on the first try.
Edmund Right. So it’s a turbocharged shortcut to familiar destinations, not a magic research brain. The article even calls out that embedding‑based retrieval still wins on rarely visited pages, which is where a lot of enterprise weirdness lives.
Geffen Exactly. If your agent mostly needs to hit known docs, account recovery guides, vendor status pages, that sixty percent first‑hit plus sub‑twenty‑millisecond server time in standard mode is compelling. For “go discover all the new regulatory updates in this niche,” you’re still in normal search‑and‑rerank land.
Edmund From a product angle, that still matters a lot. Picture a coding agent that keeps bouncing into the same docs: auth, billing, webhooks. If you can resolve those URLs almost instantly, you can afford to re‑verify every time instead of caching a summary from three months ago and praying it’s still right.
Geffen Yeah, and I do like that Zou is explicitly saying “give the agent the raw source, not just summaries of summaries.” He calls out that if you summarize, then summarize the summaries, you compound mistakes. That’s very in line with what we keep seeing: assistants need enough underlying evidence to tell current specs from outdated ones.
Edmund Okay, talk to me about the money side, because the twenty‑dollar unlimited thing is what got everyone’s attention.
Geffen So the nuance there is important. The twenty‑buck Standard plan is for individual accounts, “unlimited” Web Search plus a thousand image searches a month, and requests can queue when it’s busy. That’s great for one dev tinkering, but it’s not how you run a production agent that your sales team lives in all day.
Edmund Yeah, no one is building their company on a queued personal plan.
Geffen Exactly. For production, you’re back to real usage pricing and capacity plans. Pulse as an API is ten cents per thousand lite requests, twenty cents for standard. Web Search is one dollar per thousand, which they even frame as eighty percent off a five‑dollar list. And if you want guaranteed throughput, there’s a builder tier around two thousand ninety‑nine a month for up to fifty queries per second, plus per‑request charges.
Edmund So it’s “unlimited” as in, your side project agent can slam it without you counting tokens, but enterprises still have a budget line and an S L A conversation. That feels pretty honest, actually.
Geffen Yeah, the danger is just people conflating those. The article even lays out a price table: Parallel’s fast mode is also one dollar per thousand, Exa’s a few bucks depending on depth, Tavily is eight or sixteen. So Octen is cheap, but it’s not like everyone else forgot to lower prices.
Edmund The thing that jumped out to me more than the list prices was the Artificial Analysis Search Index numbers. Holding the L L M constant and just swapping search providers, Octen’s highlighted config scores seventy‑seven on their quality metric. Exa auto and Perplexity medium hit about eighty, Exa lite seventy‑nine, Parallel and Brave mid‑seventies.
Edmund But Octen finishes tasks in about fifteen seconds per question versus, what, nineteen for Exa lite and up to almost forty for Parallel advanced?
Geffen Yeah, roughly that. And the cost per thousand tasks: about nine bucks in search plus fourteen‑fifty in model time for Octen, total around twenty‑three fifty‑seven. Exa lite comes out over forty, Parallel near sixty. So the independent benchmark is basically saying, “Octen isn’t the absolute top on quality, but it’s pretty close, and it’s significantly cheaper and faster per completed task.”
Edmund Which is exactly the wedge they’re pitching: you might give up a couple points of answer quality on some workloads, in exchange for halving your cost per acceptable answer and shaving a few seconds off response time.
Geffen With the important asterisk that this benchmark is on Octen Search, not the ten‑millisecond Pulse routing. And it’s still a lab environment. The right metric in your own pilot is cost per acceptable answer, like the article says. If your team has to correct half the outputs, the cheap search call starts to look expensive.
Edmund Who do you think should actually care enough to go run that pilot? Because if I’m just building a one‑off chatbot, I probably don’t bother switching from whatever my L L M vendor bundles.
Geffen Yeah, if you’re doing casual chat, this is overkill. The people who should care are folks running agents that hit the web or docs a lot: procurement assistants comparing suppliers, coding agents that live in docs, research bots that pull live data. Anywhere you’ve instrumented and you can see “search and retrieval is half my latency and a big chunk of my bill.”
Edmund And especially if you’re still in that “we ration external calls” mindset. If swapping to Octen lets you go from two lookups per question to ten without blowing up costs, you can redesign the agent to be much more paranoid about checking itself. That’s the product unlock that feels real to me.
Geffen I buy that, with two more caveats. One, the self‑hosting story is not as simple as “they open‑sourced it.” The Octen Web G R four B weights and URL table are under a C C B Y N C license. So you can’t just download it and stick it behind a commercial product without talking to them. The inference code is Apache, but the model and index are non‑commercial.
Edmund So “open” like, you can tinker and benchmark, not “free infra for your SaaS.”
Geffen Exactly. And two, for truly critical workflows, I’d want to test how often Pulse actually gets the right page for my specific destinations, and how it fails. Fast wrong answers are dangerous if the agent trusts them too much.
Edmund So the move is: if you’re in that heavy‑agent bucket, carve out a pilot. Feed it your real tasks, measure end‑to‑end latency and dollars per question, and see if “verify more often because it’s cheap” actually changes what the agent can do reliably.
Geffen Yeah. If your graphs show that, then ten‑millisecond search is not just a cute number, it’s basically a new operating budget for how chatty your agent is allowed to be with the web.
Edmund And if not, you at least got a nicer benchmark out of it. Alright, I’m going to go try not to put our entire life on a twenty‑dollar unlimited plan.
Geffen Please don’t put our entire life on a queued personal account, Edmund.