Files

T

Devin AI 8e44de5271 feat(agent-proxy): segment streaming responses at ~54s to bypass Netlify Edge's ~60s response cap

Netlify Edge Functions empirically cap a single streaming response at
~60s wall-clock, regardless of activity. Confirmed against Netlify's
own canonical SSE example (edge-functions-examples.netlify.app/sse)
which also cuts at +60.1s. The Anthropic Managed Agent session is
fine for several minutes; the cap is per-HTTP-response.

This commit splits a long brief across multiple HTTP responses, while
keeping the UX of one continuous stream:

netlify/edge-functions/agent-proxy.ts
- Accept either a new-session payload {stationName, stationLocation,
  stationWebsite} or a resume payload {sessionId, lastEventId,
  startedAt}.
- On resume, skip session creation + user.message send. Just reopen
  the live SSE stream and backfill via
  GET /v1/sessions/{id}/events?after_id=lastEventId (deduped by
  event.id), then keep tailing.
- Single AbortController per segment. A 54s timer aborts the upstream,
  the for-await loops exit, and we write one final NDJSON line:
  {type:'segment_end', sessionId, lastEventId, startedAt}.
- The 20-min OVERALL_BUDGET_MS is enforced via Date.now() - startedAt
  so it spans across all segments.
- Refactor main loop so every iteration is openStream + backfill +
  tail. Cleaner than the previous initial-stream + reconnect-only-on-
  drop pattern.

src/App.jsx
- readStream() now returns a {sessionId, lastEventId, startedAt}
  payload if it saw a segment_end, or null if the stream ended
  cleanly.
- handleSubmit() loops, reopening /api/agent-proxy with the resume
  payload until readStream returns null. Spinner/status state stays
  on across segments so the UI shows one continuous stream.

README.md
- Document the segmented-streaming protocol and why it exists.

Co-Authored-By: alex <alex@semipublic.co>

2026-05-13 13:00:54 +00:00

3.2 KiB

Raw Permalink Blame History

PMC Funder Discovery Tool

A React + Vite frontend (Tailwind, Noto Serif) that collects public media station details and streams an Anthropic Managed Agent response back to the browser through a Netlify Edge Function proxy.

Setup

npm install

Add your Anthropic credentials (in .env for local dev, in the Netlify UI for production):

ANTHROPIC_API_KEY=sk-ant-...
# Optional - if you've provisioned a Managed Agent in the Anthropic console:
ANTHROPIC_AGENT_ID=agent_...
# Optional - override the model:
ANTHROPIC_MODEL=claude-sonnet-4-5

Run locally

npm run dev   # netlify dev (proxies Vite + functions on :8888)

Open http://localhost:8888.

How streaming works

The browser POSTs the form to /api/agent-proxy.
A Netlify Edge Function (Deno runtime) creates an Anthropic Managed Agent session, opens the upstream SSE event stream, and sends the user message. It tails the stream and re-opens it (with an events.list backfill, deduped by event id) whenever it drops mid session, until the session reports a terminal session.status_* event or a 20-minute wall-clock budget is hit.
Downstream to the browser the function emits newline-delimited JSON (application/x-ndjson) — text, status, heartbeat, segment_end, done, error — one object per line.
The React app reads response.body.getReader(), splits on \n, and parses each line. text lines append to the brief; status lines drive a separate “what the agent is doing now” banner.
A Thinking flag stays true until the first chunk arrives, then flips to a streaming state with a pulsing cursor.

Why segmented streaming

Netlify Edge Functions empirically cap a single streaming response at ~60 s wall-clock (Netlify's own canonical SSE example cuts at the same mark, even though the docs imply "indefinite" streaming). The Anthropic Managed Agent session itself is fine for several minutes; the limit is per-HTTP-response. So the proxy segments the stream:

Just before ~54 s the function writes one final NDJSON line — {"type":"segment_end","sessionId":"sess_…","lastEventId":"evt_…","startedAt":…} — and closes the response.
The React side sees segment_end, immediately POSTs to /api/agent-proxy again with {sessionId, lastEventId, startedAt} in the body.
The function recognises this as a resume payload: it does not create a new session or re-send the user message. It just reopens the live SSE stream, backfills via GET /v1/sessions/{id}/events?after_id=lastEventId (deduped by event.id), and keeps tailing.

The 20-minute overall budget is enforced via Date.now() - startedAt so it spans across all segments. The previous v2 Node Function (on AWS Lambda) was killed at ~27 s, well before any reconnect could fire — moving to Edge Functions extended that to ~60 s per response, and the segment loop extends it the rest of the way.

Files

src/App.jsx — form, NDJSON streaming reader, segment-end resume loop, UI states.
netlify/edge-functions/agent-proxy.ts — Managed Agent proxy with reconnect, backfill, segmentation, and NDJSON wire protocol.

3.2 KiB Raw Permalink Blame History