Devin AI
8e44de5271
Netlify Edge Functions empirically cap a single streaming response at
~60s wall-clock, regardless of activity. Confirmed against Netlify's
own canonical SSE example (edge-functions-examples.netlify.app/sse)
which also cuts at +60.1s. The Anthropic Managed Agent session is
fine for several minutes; the cap is per-HTTP-response.
This commit splits a long brief across multiple HTTP responses, while
keeping the UX of one continuous stream:
netlify/edge-functions/agent-proxy.ts
- Accept either a new-session payload {stationName, stationLocation,
stationWebsite} or a resume payload {sessionId, lastEventId,
startedAt}.
- On resume, skip session creation + user.message send. Just reopen
the live SSE stream and backfill via
GET /v1/sessions/{id}/events?after_id=lastEventId (deduped by
event.id), then keep tailing.
- Single AbortController per segment. A 54s timer aborts the upstream,
the for-await loops exit, and we write one final NDJSON line:
{type:'segment_end', sessionId, lastEventId, startedAt}.
- The 20-min OVERALL_BUDGET_MS is enforced via Date.now() - startedAt
so it spans across all segments.
- Refactor main loop so every iteration is openStream + backfill +
tail. Cleaner than the previous initial-stream + reconnect-only-on-
drop pattern.
src/App.jsx
- readStream() now returns a {sessionId, lastEventId, startedAt}
payload if it saw a segment_end, or null if the stream ended
cleanly.
- handleSubmit() loops, reopening /api/agent-proxy with the resume
payload until readStream returns null. Spinner/status state stays
on across segments so the UI shows one continuous stream.
README.md
- Document the segmented-streaming protocol and why it exists.
Co-Authored-By: alex <alex@semipublic.co>
83 lines
3.2 KiB
Markdown
83 lines
3.2 KiB
Markdown
# PMC Funder Discovery Tool
|
|
|
|
A React + Vite frontend (Tailwind, Noto Serif) that collects public media
|
|
station details and streams an Anthropic Managed Agent response back to the
|
|
browser through a Netlify Edge Function proxy.
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
npm install
|
|
```
|
|
|
|
Add your Anthropic credentials (in `.env` for local dev, in the Netlify UI
|
|
for production):
|
|
|
|
```
|
|
ANTHROPIC_API_KEY=sk-ant-...
|
|
# Optional - if you've provisioned a Managed Agent in the Anthropic console:
|
|
ANTHROPIC_AGENT_ID=agent_...
|
|
# Optional - override the model:
|
|
ANTHROPIC_MODEL=claude-sonnet-4-5
|
|
```
|
|
|
|
## Run locally
|
|
|
|
```bash
|
|
npm run dev # netlify dev (proxies Vite + functions on :8888)
|
|
```
|
|
|
|
Open http://localhost:8888.
|
|
|
|
## How streaming works
|
|
|
|
1. The browser POSTs the form to `/api/agent-proxy`.
|
|
2. A Netlify Edge Function (Deno runtime) creates an Anthropic Managed
|
|
Agent session, opens the upstream SSE event stream, and sends the
|
|
user message. It tails the stream and re-opens it (with an
|
|
`events.list` backfill, deduped by event id) whenever it drops mid
|
|
session, until the session reports a terminal `session.status_*`
|
|
event or a 20-minute wall-clock budget is hit.
|
|
3. Downstream to the browser the function emits newline-delimited JSON
|
|
(`application/x-ndjson`) — `text`, `status`, `heartbeat`,
|
|
`segment_end`, `done`, `error` — one object per line.
|
|
4. The React app reads `response.body.getReader()`, splits on `\n`, and
|
|
parses each line. `text` lines append to the brief; `status` lines
|
|
drive a separate “what the agent is doing now” banner.
|
|
5. A `Thinking` flag stays `true` until the first chunk arrives, then
|
|
flips to a streaming state with a pulsing cursor.
|
|
|
|
### Why segmented streaming
|
|
|
|
Netlify Edge Functions empirically cap a single streaming response at
|
|
~60 s wall-clock (Netlify's own canonical SSE example cuts at the same
|
|
mark, even though the docs imply "indefinite" streaming). The Anthropic
|
|
Managed Agent session itself is fine for several minutes; the limit is
|
|
per-HTTP-response. So the proxy segments the stream:
|
|
|
|
- Just before ~54 s the function writes one final NDJSON line —
|
|
`{"type":"segment_end","sessionId":"sess_…","lastEventId":"evt_…","startedAt":…}` —
|
|
and closes the response.
|
|
- The React side sees `segment_end`, immediately POSTs to
|
|
`/api/agent-proxy` again with `{sessionId, lastEventId, startedAt}`
|
|
in the body.
|
|
- The function recognises this as a resume payload: it does **not**
|
|
create a new session or re-send the user message. It just reopens
|
|
the live SSE stream, backfills via
|
|
`GET /v1/sessions/{id}/events?after_id=lastEventId` (deduped by
|
|
`event.id`), and keeps tailing.
|
|
|
|
The 20-minute overall budget is enforced via `Date.now() - startedAt`
|
|
so it spans across all segments. The previous v2 Node Function (on AWS
|
|
Lambda) was killed at ~27 s, well before any reconnect could fire —
|
|
moving to Edge Functions extended that to ~60 s per response, and the
|
|
segment loop extends it the rest of the way.
|
|
|
|
## Files
|
|
|
|
- [src/App.jsx](src/App.jsx) — form, NDJSON streaming reader,
|
|
segment-end resume loop, UI states.
|
|
- [netlify/edge-functions/agent-proxy.ts](netlify/edge-functions/agent-proxy.ts)
|
|
— Managed Agent proxy with reconnect, backfill, segmentation, and
|
|
NDJSON wire protocol.
|