I built FieldPilot — and wrote this post — for the purposes of entering the All Things Agentic Hackathon.
I've spent years implementing Dynamics 365 Field Service. The scheduling optimizer works. It respects skills, territories, time windows and priorities, and it returns a mathematically sound plan.
And every single day, dispatchers override it by hand.
Not because the optimizer is wrong — because it's answering a question that was already settled by the time it ran. Someone decided which jobs matter, how long they'll really take, and what to do when the morning falls apart. That someone is a person with a phone, reading a WhatsApp voice note from a building manager and deciding it sounds serious.
For the hackathon I tried to automate that part. This post is about how it was built — including the parts where my own measurements killed my favorite claims.
The one design decision everything else follows from
The language model does not build the route. It writes the cost function that a solver optimizes.
Judgement is a model problem: reading a voice note, looking at a photo of a leaking boiler, weighing a platinum SLA against a customer who's been bumped twice. None of that is expressible as a constraint.
Routing is a solver problem: fifty years of operations research, no reason to ask a language model to do arithmetic it will do worse.
So Gemini reads what a rules engine structurally can't — free text, audio, images — and emits exactly one number per work order: the cost of leaving that job undone today. An OR-Tools solver minimizes travel plus the penalties of whatever it can't fit. That single integer is the entire interface between the two halves. Every place a model output crosses into the deterministic side, it crosses as a named, bounded value with a deterministic fallback, and every override is logged on the work order.
Measuring it honestly turned out to be the actual project
I gave the simulator hidden ground truth: authored notes with a known "true urgency" the triage methods never see, plus an oracle that scores straight from that truth. The oracle isn't a competitor — it's the ceiling. Without it, you know method A beat method B, but not whether the gap left on the table was trivial or enormous.
That harness proceeded to kill three of my best claims.
The superadditivity claim. On my demo seed, Gemini triage plus disruption monitoring looked strongly superadditive: +21.5 points together, far more than the sum of the parts. Great story. Six seeds killed it; eighteen confirmed it dead — the interaction term is +0.3 points, which is nothing. The README keeps the prediction, its apparent confirmation, and its death on record, because that sequence is what measuring something honestly ordinarily looks like.
The severity-instability claim. On five repeated trials, adding a photo seemed to change the model's severity rating four times out of four. I wrote it up as the headline finding. At ten trials it was one in nine — indistinguishable from the model's disagreement with itself on unchanged input. I had built a tool specifically to prevent small-sample conclusions, and I made one while building it. That correction is also in the README, not quietly deleted.
The memory claim. I removed a cheat first: the solver had been reading each technician's true speed off the simulator — no dispatcher has that number. Then per-technician duration memory had to earn its corrections from completed visits. It learned the right ordering. And it was worth +3.6 ±5.2 points — statistically nothing. The devastating row is the oracle above it: perfect knowledge of every technician's speed is worth +3.0 ±6.0. There was never much on the table, and now I have the experiment that says so.
What survived
What did replicate: the model won't settle on whether a failing split unit needs hvac or hvac+elec — and three of my four simulated technicians hold hvac, so that coin flip decides who can take the job. The fix is structural: the model owns classification (identical on 10/10 repeats), and the incident taxonomy owns severity and skills. The model may raise severity — a maintenance call mentioning a burning smell is a safety call, and no lookup table knows that — but never lower it.
Five production deploys later, that split is the demo: the model's severity and confidence wobbled every time, and the dispatch came out identical five out of five, override logged.
The full system, over 18 simulated days at 48 orders each, beats its own per-seed baseline on 16 of 18 days (a coin does that less than once in a thousand), propagated lateness collapses from 368 minutes to 30, and emergency completion more than doubles — while completing fewer jobs than the baseline. It trades volume for importance, which is what it was asked to do.
Assorted scars
/healthz is a reserved path on Cloud Run — Google's frontend 404s it before your container ever sees the request. It worked locally and died in production. OR-Tools stops on a wall clock by default, so a loaded machine returns a different plan from the same inputs; counting improving solutions instead made results machine-independent and the test suite three times faster. And Firestore and Pub/Sub sat in my pyproject.toml for a week with zero imports behind them, until I accepted that declaring products you don't use is the dependency-file version of an inflated claim.
Where it all is
Everything above is reproducible from the repo — same seeds, same commands, 246 tests.
Built solo by Damian Sinay with Gemini 3.5 Flash on Vertex AI, Google ADK, Cloud Run, Cloud Storage, Secret Manager and OR-Tools, for the All Things Agentic Hackathon. #AllThingsAgenticHackathon
