migrate agent-proxy to Netlify Edge Function so long sessions stream end-to-end
The reconnect + events.list backfill in c283d88 is correct but never ran:
the previous v2 Node Function was killed at ~27 s (well before the 20 min
reconnect budget could matter), so streams always died after the first MCP
tool batch.
Move the proxy to a Netlify Edge Function (Deno runtime) which has no
streaming-duration cap as long as we keep writing to the response body.
Same reconnect / backfill / dedupe-by-event-id pattern; same NDJSON wire
protocol to the browser. Implemented with plain fetch() against the
Anthropic REST API (npm packages on Edge are beta) so we have no SDK
runtime dependency.
Frontend now POSTs to /api/agent-proxy. The Anthropic SDK is removed
from the package; @netlify/edge-functions is added for ambient types.
Co-Authored-By: alex <alex@semipublic.co>
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
A React + Vite frontend (Tailwind, Noto Serif) that collects public media
|
||||
station details and streams an Anthropic Managed Agent response back to the
|
||||
browser through a Netlify Functions v2 proxy.
|
||||
browser through a Netlify Edge Function proxy.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -31,17 +31,30 @@ Open http://localhost:8888.
|
||||
|
||||
## How streaming works
|
||||
|
||||
1. The browser POSTs the form to `/.netlify/functions/agent-proxy`.
|
||||
2. The Netlify v2 function calls `anthropic.messages.stream(...)` and wraps
|
||||
the upstream iterator in a `ReadableStream`. Each
|
||||
`content_block_delta` text chunk is enqueued as plain UTF‑8 bytes.
|
||||
3. The React app reads `response.body.getReader()` and decodes chunks with
|
||||
`TextDecoder`, appending them to the result state.
|
||||
4. A `Thinking` flag stays `true` until the first chunk arrives, then flips
|
||||
to a streaming state with a pulsing cursor.
|
||||
1. The browser POSTs the form to `/api/agent-proxy`.
|
||||
2. A Netlify Edge Function (Deno runtime) creates an Anthropic Managed
|
||||
Agent session, opens the upstream SSE event stream, and sends the
|
||||
user message. It then tails the stream and re-opens it (with an
|
||||
`events.list` backfill, deduped by event id) whenever it drops mid
|
||||
session, until the session reports a terminal `session.status_*`
|
||||
event or a 20-minute wall-clock budget is hit.
|
||||
3. Downstream to the browser the function emits newline-delimited JSON
|
||||
(`application/x-ndjson`) — `text`, `status`, `heartbeat`, `done`,
|
||||
`error` — one object per line.
|
||||
4. The React app reads `response.body.getReader()`, splits on `\n`, and
|
||||
parses each line. `text` lines append to the brief; `status` lines
|
||||
drive a separate “what the agent is doing now” banner.
|
||||
5. A `Thinking` flag stays `true` until the first chunk arrives, then
|
||||
flips to a streaming state with a pulsing cursor.
|
||||
|
||||
Edge Functions are required here — the previous v2 Node Function ran on
|
||||
AWS Lambda and got killed at ~27 s, well before any reconnect could
|
||||
fire. Edge Functions run on Deno Deploy with no streaming-duration cap
|
||||
as long as the function keeps writing to the response body.
|
||||
|
||||
## Files
|
||||
|
||||
- [src/App.jsx](src/App.jsx) — form, streaming reader, UI states.
|
||||
- [netlify/functions/agent-proxy.js](netlify/functions/agent-proxy.js) —
|
||||
Managed Agent proxy with `ReadableStream`.
|
||||
- [src/App.jsx](src/App.jsx) — form, NDJSON streaming reader, UI states.
|
||||
- [netlify/edge-functions/agent-proxy.ts](netlify/edge-functions/agent-proxy.ts)
|
||||
— Managed Agent proxy with reconnect, backfill, and NDJSON wire
|
||||
protocol.
|
||||
|
||||
Reference in New Issue
Block a user