Forward Deployed Agent Workflows at Enterprise Scale

The constraint

The client had a proof of concept that worked in a notebook and failed everywhere else. Prompt chaining had grown organically: no way to test a change, no way to explain an output to a reviewer, and no rollback story. The ask was not "make the model better" — it was "make this something our own team can operate after you leave."

That last clause is what defines forward deployed engineering. You are not shipping a demo; you are shipping an operable system plus the understanding to run it.

Approach

The first decision was to reject open-ended agent autonomy. A ReAct-style loop that decides its own next step is elegant in a paper and untestable in an audit. Instead the workflow became a SequentialAgent: a fixed pipeline of narrowly scoped sub-agents, each with an explicit tool allowlist and a typed output contract.

Rejected alternatives, and why:

Single mega-prompt. Cheapest to build, impossible to debug. One bad output and you have no idea which instruction failed.
Fully autonomous multi-agent graph. Impressive in a demo. Non-deterministic cost, non-deterministic latency, and no reviewer will sign off on it.
Kubernetes-hosted agent mesh. The client had no platform team to run it. Cloud Run scale-to-zero was the honest answer.

Architecture

A root orchestrator receives the request, validates it against a schema, then passes through sub-agents in a fixed order: extract → enrich → reason → format. Each stage writes its intermediate output to Firestore keyed by session, which gives you two things almost for free: a resumable workflow and a complete audit trail.

Serving is Cloud Run with min-instances=0 and concurrency 40 — the workload is IO-bound on the model call, so one container handles many in-flight requests cheaply. Cloud Build produces the image; Artifact Registry stores it; Terraform provisions all of it from an empty project.

What broke

Cold starts. With min-instances=0, the first request after an idle period took six seconds — Python runtime plus Vertex AI client initialisation. Users read that as broken, not slow.

Two fixes, in order of cost-effectiveness. First, the UI began streaming a thinking indicator immediately, which reframed the wait as work rather than failure. Second, a Cloud Scheduler job pinging the health endpoint every ten minutes during business hours kept one instance warm for a fraction of what min-instances=1 would have cost.

The deeper lesson: perceived latency is a UI problem at least as much as an infrastructure problem.

Outcome

Pilot in production eleven days after the scoping call. Every agent decision traceable to a stored intermediate state. The client's own engineers took over operations after a two-session handover, which was the actual success criterion.