Building AskZO: How Runzo's AI Food Concierge Gets Smarter Every Conversation

The problem with ordering food in 2026

Ordering food should be simple. It isn't. The average person opens a delivery app, stares at 200 restaurants, scrolls for six minutes, and either picks the same thing they always order or abandons the search entirely. A delivery app is a catalogue. AskZO is a conversation.

The ambition was straightforward: replace the browse-and-tap loop with a natural language exchange. "I want something good for lunch near me." "Catering for 40 people, Indian food, Thursday." "Can I get that place I ordered from last week?" AskZO understands all of these, calls the right tools, and gets the job done — without the user knowing anything about API calls or menu IDs.

Building that required more than a language model with a few functions attached. It required an architecture that remembers users across sessions, learns from every conversation, grounds every factual claim in a live data source, and surfaces the right prompt at the right moment before the user has typed a word. This is the engineering behind it.

The Runzo Agentic Platform

AskZO runs on RAP — the Runzo Agentic Platform — a Bun/Fastify service at the centre of all AI interactions across the product. RAP is intentionally decoupled from the Runzo backend. It speaks to Runzo through a connector interface, the same way it could speak to Square, Clover, or any other platform. Runzo is RAP's first tenant, not its only one.

The platform follows a supervisor-and-specialist model. A single consumer-facing agent slug — askzo-consumer — receives every user message. A lightweight intent classifier reads the first few tokens and routes the conversation to one of two specialists:

• ask-instant: restaurant discovery, menu browsing, cart management, reservations
• catering-request: formal catering event intake, caterer shortlisting, bid submission

The supervisor handles the routing decision transparently. The user sees one conversation with one agent. Internally, different specialists handle different domains — each with their own tools, system prompt, and context.

Every agent turn runs through an agent loop that: loads context, calls the LLM with available tools, executes tool calls against the Runzo backend, streams the response to the client via SSE, and runs post-turn evaluation non-blocking. The loop supports up to 15 iterations per turn — enough for multi-step tasks like "find me a Thai restaurant near downtown, check their menu, and add the pad thai to my cart" in a single response.

Three-layer memory

The DoorDash engineering team described their memory architecture as having three layers. We built the same model for AskZO, with some implementation differences.

Layer 1: Long-term episodic memory

After every session completes, a non-blocking async process fires. It takes the last 20 messages from the conversation, sends them to Claude Haiku with a 15-second timeout, and asks for 3–5 reusable facts about the user: dietary preferences, party size patterns, preferred cuisines, recurring event details.

These facts are stored in a UserMemory table (PostgreSQL, via Prisma) with an importance score of 0.5. At the start of every new session for the same user, the top 5 most important memories are recalled and prepended to the system prompt under a [MEMORY — facts recalled about this user] header.

The result: a user who mentioned they prefer halal food in January will find that AskZO already knows this in February without being told again. A user who regularly orders for 15-person team lunches gets that context carried forward automatically.

Layer 2: In-session context

Within a single conversation, session state lives in Redis. Every message, tool call result, and cart state is preserved across the full turn sequence. A topic shift detector (dynamicMemory flag) compares each new user message against prior conversation history. When it detects a shift — say, the user pivots from "find me a restaurant" to "actually I need catering for next week" — it re-anchors the system prompt context before the next LLM call.

Layer 3: Entity memory

For agents acting on behalf of a business entity (a restaurant, an event organiser, a recurring corporate account), a separate EntityMemory table tracks operational patterns at the entity level. A restaurant that responds to bids with specific preferences, or a corporate account that always orders for the same office address, builds up an entity-level profile that informs every session linked to that entity.

The learning flywheel

Static few-shot examples get stale. The most useful examples for an agent handling a Tuesday lunch order are Tuesday lunch orders that went well — not hand-curated examples from a product brief.

RAP's learning loop works as follows: after a session ends, an effectiveness scorer computes a 0–1 score based on outcome (order submitted = 0.85, completed = 1.0, planned = 0.65, abandoned = 0), session efficiency (penalising long turn sequences), and error rate (guardrail hits, tool errors). Sessions scoring above 0.72 have their best 2–6 turns extracted and stored as FewShotExample records in PostgreSQL with a pgvector embedding.

At the start of every new session, a semantic similarity search against the current user message retrieves up to 3 of these examples and appends them to the system prompt as style and approach guides. The pool is capped at 200 examples per agent, with the lowest-scoring ones pruned when the cap is hit. The examples are always real conversations — the agent learns from what actually worked, not from what we predicted would work.

Grounding: never hallucinate a menu item

Grounding was the first production failure category we anticipated. A language model will confidently state that a restaurant is open until 10pm, or that a dish costs $12, if that information was in its training data or simply plausible. It doesn't know it's wrong. The user does, when the reality doesn't match.

We apply grounding at two levels.

Prompt-level prevention

Every consumer agent system prompt ends with an explicit GROUNDING RULE:

Every factual claim you make — restaurant name, hours, rating, price, menu item, availability, contact detail — must come from a tool result returned in this conversation. Never state a fact from training data or prior knowledge. If you have not called a tool to verify something, say you will look it up rather than guessing.

This rule is appended via a database migration, not hardcoded in application code. Updating it across all agents is a SQL update, not a deployment.

Post-stream detection

After every response is streamed (post-hoc, since SSE responses can't be blocked mid-stream), a two-layer grounding checker runs. The first layer extracts dollar amounts from the response and cross-references them against a token set built from tool results in the current turn. The second layer sends the full response text and a summary of tool results to Claude Haiku with a 5-second timeout and asks it to identify suspicious factual claims. Results are logged to an AuditEvent table.

The post-hoc layer doesn't prevent a grounded response from being delivered, but it builds a record of drift that informs future prompt tuning. The combination of prompt-level discipline and post-stream auditing has kept grounding failures near zero in production.

Contextual entry points

One insight from DoorDash's engineering post resonated strongly with us: the first moment a user sees the AI interface is as important as what happens inside it. A blank input box is a cold start. A blank input box with three chips that say exactly the right thing for this moment is an invitation.

We built a GET /agents/:agentId/suggestions endpoint that returns 3 context-aware prompt chips based on time of day and agent type. The logic is simple in v1 — meal-window bucketing (breakfast 6–11am, lunch 11–3pm, afternoon 3–6pm, dinner 6–11pm) with catering-specific chips for the catering agent — but the architecture is in place for v2 to incorporate order history and user preference signals. Results are cached 5 minutes in Redis, keyed by agent and user ID.

These chips appear when the AskZO panel opens, before the user types anything. Tapping one sends that text directly as the first message. The user never sees a blank start.

Artifacts: structured outputs that survive turns

A conversation is ephemeral. A caterer shortlist that a user spent five minutes reviewing should not be. We built a versioned artifact pattern to solve this.

When an agent assembles a structured result — a list of caterers for a catering event, a set of menu items ready for the cart, a full event plan — it calls create_artifact. This writes a document to the AgentArtifact table (PostgreSQL) with a stable UUID, the artifact type, a title, and a JSON data payload. The version starts at 1.

The widget receives an artifact SSE event and renders the artifact as a card in the chat — formatted based on type: caterer shortlists become bulleted lists with rationale, menu selections show items with quantities and prices, event plans show a compact summary line.

At the start of any subsequent turn, the system prompt is augmented with an ACTIVE ARTIFACTS block containing the current state of every active artifact for the session. The agent is instructed to call read_artifact at the start of any turn where the user refers to a previous list or plan — picking up any edits the user may have made between turns via the PATCH /artifacts/:id endpoint.

This is the pattern that makes AskZO feel less like a chat session and more like a persistent workspace. The agent and the user are working on the same document together, across turns, with the agent able to read and update the user's edits.

What we learned building this

A few things surprised us:

• The most impactful single change was enabling the memory flags. The episodic store, the learning flywheel, the grounding checker — all of it was built and sitting inert with all flags set to false. One database migration turned on the entire intelligence layer for three agents simultaneously. Infrastructure is only as valuable as its activation.
• Prompt-level grounding outperforms post-hoc detection by an order of magnitude. An LLM that is explicitly told "you must cite a tool result for every factual claim" makes far fewer grounding errors than one that is simply checked after the fact. The check is a safety net, not a substitute for clear instruction.
• Few-shot examples should be earned, not authored. We spent time writing example conversations for the few-shot injection layer before realising we could use the production conversations that users had already confirmed were good. Real examples are more diverse, more accurate to actual user language, and self-maintaining as the user base grows.
• The first message matters as much as any other. A contextual suggestion chip that matches what a user was actually about to ask is not a nice-to-have. It changes the completion rate for the whole session. Users who tap a chip complete a task at meaningfully higher rates than users who start from a blank input.

What's next

We're working on three things in this space.

The first is deepening the personalized suggestion layer. Time-of-day bucketing is a blunt instrument. Order history signals — "you ordered from this restaurant three weeks ago, you haven't been back since" — are sharper and more actionable. The endpoint architecture is in place; the signal enrichment is next.

The second is the artifact editing interface. Right now the PATCH /artifacts/:id endpoint exists, and the widget renders artifact cards. The user-facing editing UI — being able to remove a caterer from a shortlist by tapping a button, or change a quantity in a menu selection — is the next widget deliverable.

The third is expanding RAP beyond Runzo. The connector interface was designed from day one to support multiple platforms. The next tenant using RAP's intelligence layer will be a very different domain. AskZO is Runzo's interface. The platform underneath it belongs to a broader vision.