Hidden City: RavenDB as In-Cluster Working Memory for LLM Agents
Overview
This project solves the hidden cost problem in LLM agents by keeping the agent's working memory in the same cluster as the model call. Instead of stuffing raw API responses into every prompt, the agent queries RavenDB, co-located in Kubernetes, as an explicit tool call. The LLM only ever sees a pre-structured, token-budgeted result, never the raw payload.
The application also demonstrates conversational hidden-city flight detection. Airlines price connecting routes cheaper than direct legs; the agent compares cached and live prices from Travelpayouts, scores the savings against a risk profile, and surfaces the opportunity as information only, never automating a booking.
To fully embed itself in the Kubernetes ecosystem, the RavenDB cluster is declared as a RavenDBCluster custom resource and reconciled by the RavenDB Kubernetes Operator, which handles bootstrap, certificate wiring, and rolling node upgrades. On top of this, a CronJob bulk-ingests Travelpayouts prices every 6 hours, and a RavenDB Data Subscription pushes price-drop alerts to a worker with no polling loop.
The business value is cost, made visible. Every retrieval that stays inside the cluster is a retrieval that never crosses the egress boundary and never inflates the token payload billed by the inference provider — the demo costs each of the three architecture stages live, from a naive ~40k–100k tokens per call down to a ~2,500 token budget enforced end to end.
Features used
The model decides what to retrieve and when. Nothing is blindly pushed into the prompt: search_routes, get_live_prices, get_user_profile, and update_constraints are tool definitions the model calls explicitly, and only their (token-budgeted) results re-enter the conversation.
tools = [search_routes, get_live_prices, get_user_profile, update_constraints]
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=history,
tools=tools,
)
One narrow, documented exception pre-resolves unambiguous "from X to Y" messages and pre-fetches search_routes itself, cutting a turn from two OpenAI calls to one (~2,100 tokens down to ~750 measured) — it only fires when both airports resolve to exactly one match each, and falls back to the model unchanged otherwise.
Airport city/IATA lookup runs against a RavenDB auto-index RavenDB creates on first query — no index definition is checked into the codebase.
session.query(object_type=Airport) \
.search("name", query_text) \
.first()
Airport documents carry a location_vector field. When search_routes finds no direct or connecting route, it queries that field to find geographically nearby airports instead of relying on a hand-curated list — one document type, no separate vector database.
session.query(object_type=Airport) \
.vector_search(lambda f: f.with_field("location_vector"), origin_vector) \
.take(5)
Route documents are refreshed hourly by ETL and stamped with a short TTL on their price fields, so stale prices fall out of query results automatically instead of needing a scheduled cleanup task.
Binary user data lives as RavenDB attachments on the users/... document — not queryable, not indexed, uploaded and retrieved whole via the Profile screen, separate from the get_user_profile tool the agent calls.
A RavenDB Subscription pushes price changes straight to the worker process. There is no polling loop and no Kafka topic to stand up just to move a price delta from the database to a notifier.
with store.subscriptions.get_subscription_worker(subscription_name) as worker:
def process_batch(batch):
for item in batch.items:
notify_price_drop(item.result)
worker.run(process_batch)
Conversation state is a RavenDB document, not pod RAM or a Redis sidecar. active_constraints (carry-on only, max stops) are indexed fields, not buried in turn text, so the agent loads only the last N turns plus constraints at the start of each turn — never the full raw history. Killing the agent pod mid-conversation and resuming is part of the demo.
The agent, worker, and scraper all connect to RavenDB over mutual TLS, wired automatically by the RavenDB Kubernetes Operator's certificate handling.
Technologies
- RavenDB 7.2
- Python 3.11–3.13
- FastAPI + Uvicorn
- OpenAI API (
gpt-4o-mini) - Kubernetes + RavenDB Kubernetes Operator
- Docker / Docker Compose
- kind, Helm, kubectl (local Kubernetes)
- uv (Python package/venv manager)
Run locally
- Check out the repository.
- Install prerequisites: Python 3.11–3.13, uv, and Docker (Docker Desktop or Docker Engine) for local mode; Kubernetes mode additionally fetches kubectl, Helm, and kind automatically.
- Windows: run
.\start.ps1and pick Local or K8s mode when prompted (or pass-Mode Local/-Mode K8sdirectly). macOS/Linux/WSL2, Kubernetes mode: runbash k8s/start-k8s.sh. - When prompted, provide an
OPENAI_API_KEY(required). ATRAVELPAYOUTS_TOKENand a RavenDB license are optional — without them the app falls back to fixture data and RavenDB Developer Mode respectively. - Once running, open the agent's
/setupwizard athttp://localhost:8001/setupto add any key you skipped, and RavenDB Studio athttp://localhost:8080(Local mode) orhttps://localhost:8081(Kubernetes mode) to inspect the enrichedRoutescollection.
Notes
Every demo scenario runs end to end on fixture data seeded locally, so no live Travelpayouts API dependency is required to see the full flow. This project demonstrates the detection of hidden-city pricing patterns for educational purposes only — it does not automate booking, generate itineraries, or interact with airline reservation systems.
Community & Support
If you spot a bug, have an idea, or want to ask a question, open an issue or pull request in the repository. The RavenDB team also hangs out on the RavenDB Discord server.
Contributing
Contributions are welcome. See the repository's CONTRIBUTING file for details.
License
This project is licensed with the MIT license.
Challenges & Solutions
Feature
Category
Travel