The same tools, driven two ways: a workflow where code decides, and an agent where the model decides. Both measured on the same 75 questions.
You have 50 minutes at Zurich. The airport shows a walking time to the gate. It does not tell you that you will cross passport control, that your Bangkok arrival means a second security check, or that the lounge you're entitled to is on the wrong side of the border. You find out by walking.
Ask a chatbot with the airport's pages pasted in and you get a fluent paragraph. Ask it 25 different ways and the paragraph changes its mind about whether there is a passport control at all. That is the failure this project is about: the answer needs to be computed from a structure (which zone am I in, which zone am I going to, which rules fire on that crossing), and text retrieval does not compute.
Only one step needs a language model: reading the question. "I land at D33 from Bangkok, fly out from A63, 50 minutes, I'm Star Gold" has to become {arrival: D33, departure: A63, minutes: 50, origin: TH, status: star_gold}. That is fuzzy input and a small model (Claude Haiku 4.5) does it for a few hundred tokens.
Everything after that is deterministic. Gates map to gate areas. Gate areas have a Schengen zone. A crossing between zones is a passport control. An origin country is on the EU one-stop-security list or it isn't. The shortest route is a graph search. The verdict is arithmetic against a minimum connection time. None of that should be left to a model, and in this build none of it is: the answer text itself is a template over what the workers returned, so the model cannot introduce a fact that was not retrieved.
| Decision | Alternative | Why, and the cost |
|---|---|---|
| Zurich only | Frankfurt, the bigger hub | I live in Zurich and can check routes on my own flights. Frankfurt I can't ground-truth. Cost: one airport is a demo, not a product. |
| Hand-built graph (22 nodes, 27 edges) from public pages and OpenStreetMap | Indoor-map vendor API, or OSM routing | No public floor-plan API exists for ZRH. OSM has every gate number and level but no corridors. Cost: walking times are my estimates from gate geometry, marked as such. |
| Workflow first: explicit orchestrator, four workers, run once | An agent loop with tools from day one | The workflow pins the tools and the expected answers, so the agent can be measured against it. Both are built; the agent is the row that costs 24 to 28 points (below). Cost: nothing re-plans yet. |
| Deterministic composer | LLM writes the answer from the trace | The validator can prove every route edge exists; it can't prove a paragraph. Cost: the prose is plain. |
| Same documents for all three systems | Give the baseline "fairer" documents | The comparison is the project. Cost: the baseline gets tables rendered as text, which is exactly the point. |
| Public data only, every number with a URL | Use what I know from work | I work in this industry. If I can't cite a public page it doesn't go in. Cost: the minimum connection time is the weakest source in the corpus (below). |
Ambiguous and out-of-scope questions ("gate 34 to gate 55", "will my flight be on time?") never reach a worker. The trace shows the orchestrator's reason and an empty worker list, which is the honest answer.
This version is not a loop. There is no re-planning, no repeated tool call, no memory. Each worker runs at most once, in a fixed order, and the result is validated once. I built it this way first so that every tool existed, every expected answer was pinned, and debugging on day three meant reading a trace instead of a transcript. Then I handed the same tools to the model and let it decide. That is the next section.
The two comparison systems share the prompt and the documents. Naive vector RAG: chunk the four rule documents and the two csv tables, embed with bge-small, cosine top-6, one model call. Hybrid + reranker: BM25 and dense retrieval fused by reciprocal rank, a cross-encoder reranks twelve candidates to six, same prompt. Neither has the graph.
The workflow above scores 100% because code decides what to check. That is also its limit: the orchestrator handles the eight question shapes I wrote branches for, and nothing else. So the second build hands the same five tools to the model and lets it decide what it needs, in what order, and when it is done. Nothing else changed. The tools are the same functions. The validator re-checks whatever the model submits against the graph. Citations are derived from the tools it actually called. The answer text is the same template. The model can choose badly; it cannot fabricate.
| system | golden 25, all four right | held-out 50, all four right | model calls per question |
|---|---|---|---|
| workflow (code decides) | 100% | 100% | 1 |
| agent (model decides, same tools) | 76% | 72% | about 3 |
| failure, across all 75 questions (20 missed) | count | what happened |
|---|---|---|
| Skipped the checkpoint-rules tool on a non-Schengen arrival | 9 | Both gates were non-Schengen, so the model reasoned "no border, nothing to check" and never asked whether the origin requires re-screening. The validator flags it every time. |
| Put a B-numbered gate on the Schengen side for a non-Schengen arrival | 3 | The lookalike trap. The workflow handles it in code; the model had the rule and did not apply it. |
| Called a connection below the minimum connection time "yes" or "tight" | 3 | The tool returned 40; the minutes were below it; the model softened the verdict anyway. |
| Called an 11-minute margin "tight" | 1 | The rule (under 10 minutes is tight) was in its instructions. |
| Wrong behaviour: declined instead of answering or asking, or asked about Frankfurt instead of declining | 3 | |
| Submitted an empty route; cited the wrong rule set with the right route | 2 |
The pattern is the point. The model never gets a queue time or a sum wrong, because tools do those. It gets decisions wrong: it skips a check that looks unnecessary, it softens a verdict the rule made hard, and once in a while it sends the route tool to the wrong area. Those are exactly the decisions the workflow makes in code. The validator catches every one of them, which is what a validator is for.
On the record: after the first run I fixed one thing in the loop, not in the prompt. The model sometimes ends its turn in prose ("which dock is gate 32 in?") instead of calling the submit tool, and the first run counted that as a failure. The loop now reads such prose as a clarification request or a decline by its wording; eleven rows were re-mapped without re-querying the model. The agent's instructions were not changed after the first run, so its numbers are first-attempt numbers.
A common framing says an agent suits flight connections because they are fluid: delays, gate changes, queues. The shape of that argument is right and the details usually are not. Two things it gets wrong about this build: the workflow does not stop at the minimum connection time (it computes the walk and the queues and says "tight" at 39 minutes out of 40), and the Skymetro and the passport crossing are in the graph, so both systems handle them. And the things an agent is credited with, live queue lengths, a tarmac shuttle, a rebooking, are properties of having live data and authority, not of being an agent. A workflow with a queue feed would use it just as well.
The honest version, for this product, is three conditions. An agent earns its place when one of them holds.
| condition | what it needs | status here |
|---|---|---|
| The question has a shape the workflow has no branch for ("is there time for the lounge and dinner?", "my flight stops in Chicago first") | A model that decides what to check, and tools that can be combined freely | Built. This is the agent row. Cost measured: 24 to 28 points on questions the workflow was written for. Untested on questions it was not. |
| The world changes after the first answer: a delay, a gate change, a longer queue | Live flight, gate and queue data, and a loop that re-runs the affected tools when they change | Mechanism built and demonstrated with simulated events (next section). The data is not public. This is the layer United ships. |
| The right response is an action, not an answer: hold the connection, rebook, alert staff | Authority and integrations with the airline | Out of scope by design. The airline does this; a passenger tool should hand over cleanly. |
So the answer to "why move from a workflow to an agent" is not "because travel is unpredictable". It is: the agent is the mechanism that makes re-planning possible, the workflow is the reference that lets you measure what the mechanism costs, and the eval says what that cost is today. When the live data arrives, the loop is already written and already scored.
The goal is that the answer does not go stale. Live flight, gate and queue data are not public, so the demo stands in for the feed with three buttons under every agent answer: my inbound is 15 minutes late, departure gate changed, passport queue is 25 minutes. Pressing one sends the original question, the earlier answer, and the event to the agent. Nothing tells it which tools to re-run. It decides, and the answer opens with what changed: "verdict yes to no; minutes available 50 to 35." The page says the events are simulated, because they are.
It is scored the same way as everything else. Twelve cases: four answered questions, three events each. The expected answer is the workflow run with the event applied (fewer minutes, a new gate, or one checkpoint's queue overridden), so the reference is the same code that scores 100% one-shot.
| re-planning, 12 simulated events | verdict right | route right | both | model calls per event |
|---|---|---|---|---|
| agent, given the question, its earlier answer, and the event | 83% | 92% | 75% | 2.4 |
What it got right: every gate change produced a new route through the right checkpoints; every queue event was re-costed by re-running the route tool and nothing else; a delay that dropped the time below the minimum connection time flipped the verdict to "no" for the right reason. The three misses are the same decision failures as before: twice, on a delay, the model softened the verdict to "tight" without re-checking the minimum connection time (once it should have been "no", once "yes"), and once it sent the route to the wrong gate area. Every miss was flagged by the validator.
On the record, two interface fixes made while building this and kept for all rows: the "minutes available" field was described more precisely after the model put the spare minutes in it, and the summary of the earlier answer now states outright whether passport control and re-screening applied, after the model read "rule consulted: re-screening" as "re-screening required" and added a loop that was not there. The one-shot agent rows were re-run with the same code and moved from 68% and 68% to 76% and 72%; some of that is the clearer field and some is run-to-run variance in the model, which is itself a finding. The model's instructions were not otherwise changed.
What would close the gap, in order and each measurable against the same 75 questions: force the checkpoint-rules call whenever the arrival is non-Schengen; send a failed validator check back to the model for one retry; and let the model ask the traveller only when the missing fact would change the answer. Try the agent on the demo: pick "agent" and read the steps.
The airport is 22 places joined by 27 walks. Gates are grouped into areas, because walking times only make sense at that scale: there is no node for A63, there is one for "Dock A gates A62 to A86". Each place has a zone, Schengen or not. Passport control is a place with a 10-minute queue, and it is the only way across the border. Security re-screening is a short loop you leave the pier for and come back from, taken only when the origin country requires it.
Two things in this picture do the work. Checkpoints are places, not labels. If a route crosses the border, the passport node is on the path and its queue is in the total; the system cannot forget a checkpoint the way a text system can, because the checkpoint is part of the geometry. B and D are one pier. Gate B34 and gate D34 are the same door, named B for a Schengen flight and D for a non-Schengen one, so the graph has two nodes for one pier with passport control between them. "Land at D34, depart from B34" is the same gate and still needs passport control. That case shaped the graph before a line of code was written.
"Which sentence in the documents looks most like this question?" No sentence says D33 to A63 crosses the border. There are thousands of gate pairs and the airport's page describes zones, not pairs. The model has to infer it, and across 25 questions it infers it differently each time.
"Which area is D33 in? Which zone? Which area is A63 in? Which zone? Do they differ?" Three lookups and a comparison. Then a shortest-path search whose answer is the same every time. The rules documents are read by section heading, so the passage cited is always the one that applies.
25 questions, 8 traps. 19 expect an answer; 6 expect the system to ask or decline.
Expected answers were derived from the graph and the rules on day one, before either system ran. Two caveats I want on the record. First, the draft questions used gate numbers from memory; most of them don't exist at Zurich (there is no A60, for instance). The planner maps any number to a gate area, so the eval still works, but the questions will be renumbered against the real gate list once I've walked it. Second, because the expected routes come from the same graph the planner searches, a perfect score proves that the question-reader extracts the right facts and that the pipeline is internally consistent. It does not prove the graph matches the building. That check happens on foot. Third, the question-reader's instructions were fixed three times against this set, which is why there is a second, held-out set below.
"Route exact" is strict on purpose. A route with the right destination that skips the security loop is wrong, because the minutes are wrong and the traveller would not be warned. All three systems return the same answer shape, so one scorer compares all three to the same expected columns.
| system | answered / asked / declined right (25) | verdict right (19) | route exactly right (19) | rules cited right (19) | all four right |
|---|---|---|---|---|---|
| naive vector RAG | 56% | 21% | 5% | 11% | 28% |
| hybrid search + reranker | 68% | 16% | 5% | 21% | 24% |
| graph + orchestrator (workflow) | 100% | 100% | 100% | 100% | 100% |
| agent: same tools, model decides | 100% | 84% | 95% | 79% | 76% |
The question-reader is the only learned part, and its instructions were tuned on the 25 above. So the 25 were reworded 50 times (typos, lowercase gates, "A10 -> A85", "half an hour", "LHR-ZRH-LIS") after the tuning stopped, and never shown to it before scoring.
| system | answered / asked / declined right (50) | verdict right | route exactly right | rules cited right | all four right |
|---|---|---|---|---|---|
| naive vector RAG | 50% | 16% | 8% | 8% | 24% |
| hybrid search + reranker | 54% | 18% | 8% | 16% | 24% |
| graph + orchestrator (workflow) | 100% | 100% | 100% | 100% | 100% |
| agent: same tools, model decides | 94% | 89% | 89% | 71% | 72% |
Two caveats stay on the record. The expected routes come from the same graph the planner searches, so both 100% rows prove internal consistency and a robust question-reader, not that the graph matches the building. And the rewordings were written by me, so they cover the phrasings I could think of, not the ones travellers will type.
| failure mode | naive | hybrid + reranker |
|---|---|---|
| Hedged: asked for the dock letter when the question already gave it ("which dock is D50 in?") | 11 | 8 |
| Route wrong or partial | 4 | 7 |
| Missed a passport control that the route crosses | 3 | 3 |
| Verdict flipped (mostly yes → no) | 4 | 8 |
| Invented node ids that don't exist | 0 | 0 |
Two things surprised me. The naive system almost never hallucinated: it never invented a node, and its favourite failure was refusing to commit, because the retrieved chunk about the shared B/D pier made a plain "D50" look ambiguous. And better retrieval made the hedging better and the reasoning worse: the reranked system answered more often, cited the right rules more often, and flipped more verdicts, because it now had enough numbers in context to do the arithmetic wrong. Retrieval quality was not the bottleneck. Computation was.
The graph system missed three questions on its first run, all in the one place a model is used. "Both non-Schengen" was read as an unknown non-EU origin (so a re-screening appeared that shouldn't); "a one-stop-security origin" was read the same way; and "Land A40, depart A45" lost its letters. Three sentences added to the extraction instructions fixed all three. The baseline was not touched.
Cost: one full run of all three systems on 25 questions is about 15 cents. The graph system uses a quarter of the input tokens of the baseline, because the model only reads the question.
Live position, turn-by-turn, reading the sign in front of you, audio. That is a mobile app on top of indoor positioning, and vendors already sell the positioning (MappedIn, MapsPeople, Navigine sell wayfinding SDKs to airports; Frankfurt has run WiFi-based indoor routing since 2015). If one of those APIs were available it would replace `edges.csv` and nothing else. The rules layer on top, will I make it and what will I hit, is the part nobody exposes to passengers, and it is the part this build is about.
One Railway service (FastAPI, Docker, embedding and reranker models baked into the image) serves the API and these pages. Haiku 4.5 at $1 per million input tokens; a graph answer costs well under a tenth of a cent, a naive answer a few tenths. Eval runs are cached per system, model and corpus hash, so re-running is free until the data changes. Limits: static data, one airport, English only, no accounts, no live anything. What breaks first: the airport renumbers a gate area, and nobody tells the csv.
Disclaimer: Make My Gate is a personal portfolio project built on public data. Walking and queue times are estimates. Minimum connection times are set by airlines and airports and change without notice; always verify with your airline or travel agent. Not affiliated with Zurich Airport or any airline. Not responsible for missed connections: check your boarding pass and the signs.