I recently needed my front-end to know when a data export was done. It's a simple csv or xlsx file, and unless the user has heaps of data, it takes a few seconds. The server just has to tell the client "your file's ready". No progress bar, no cancel button.
Out of bad habit, my first thought was a status endpoint the client could poll (known in the hood as short-polling). I've built countless of these, and they seem to be the favourite way for dev teams to rack up tech debt they never pay off.
There are a few well-known options here, and it's mostly about picking the right one for your case. And if you ask what’s the purpose then, it’s because every time I do one of these light-hearted write-ups, I end up in deep and fantastic conversations with fellow nerds.
So, onward. Short-polling takes a minute to set up: you ask the job, on an interval, whether it's finished. The server answers, the connection closes, and you repeat until it fails or succeeds.
It's the developer's version of "Are we there yet?". Most of those requests are worthless. They burn server cycles and bandwidth, and open the door to state management bugs you didn't need.
Interval is basically impossible to get right - too slow and the updates feel choppy, too eager and you're bloating the network. That said, for long-running jobs where nobody needs an instant update, a well-spaced interval is perfectly fine.
Long-polling is the same idea, slightly more sophisticated. Plain HTTP but the server holds the connection open until there's an actual update or it times out after ~30s.
Handling reconnections gracefully and timing them with a margin before the timeout can be a bit of a pain. Keeps threads captive on synchronous servers, which isn't ideal. Still, unless you have high-frequency updates, I'd pick it over short-polling. It has served devs well for a long time.
Now, the real deal: WebSockets. They start life as an HTTP request then upgrade to their own protocol over TCP. That's why proxies and firewall configs can be a bit of a PIA (pardon my french), but in return you get proper bidirectional streaming.
Either side sends messages whenever it wants, with only a few bytes of framing on each one. It's quick, but trade-offs are extra complexity on the infrastructure side.
Unless you really need low-latency, two-way streaming, think collaboration tools or multiplayer anything, they're overkill.
For my simple stuff, I went with Server-Sent Events. Same push-from-the-server idea, way easier to set up, and no load balancer drama.
It's still plain HTTP. The client opens a connection with the EventSource API, and the server streams text events the moment they happen. The browser handles reconnection for you and keeps track of the last event it received.
On the server, you return a streaming response with the Content-Type set to text/event-stream, and send each event as your generator yields it. The client gets them in real time instead of waiting for the function to finish.
@app.get("/exports/{job_id}/events")
async def export_events(job_id: str):
async def stream():
async for status in watch_export(job_id):
yield f"id: {status.seq}\nevent: status\ndata: {status.json()}\n\n"
return StreamingResponse(stream(), media_type="text/event-stream")Then…
const events = new EventSource(`/exports/${jobId}/events`);
events.addEventListener("status", (e) => {
const status = JSON.parse(e.data);
if (status.done) events.close();
});If you're on a Python back-end, it's dead simple, as most frameworks have you covered out of the box. On FastAPI, look at StreamingResponse (thx Starlette), on Flask, stream_with_context, and there are plenty of others.
My events are just plain text with data, event and id fields. The id is what makes reconnection easy: the browser sends it back as Last-Event-ID when it reconnects, so the server can resume from where it left off.
Know your options, you could open an envelope with a chainsaw too but it's prlly not the most efficient way to do it.