FastAPI Background Tasks vs asyncio.create_task vs Redis Queue

A POST endpoint returns 202 in 80ms. Somewhere off the request path, an email gets sent, a thumbnail gets resized, an embedding gets written to a vector store. Three tools handle that "somewhere" in the FastAPI ecosystem, and picking the wrong one shows up as either lost work or a dead worker process at 3am.

This compares BackgroundTasks (FastAPI's built-in), asyncio.create_task (the raw asyncio primitive Starlette wraps), and a Redis-backed job queue (arq is the canonical async-native choice). The decision tree is shorter than the documentation suggests, and most of it comes down to one question: what happens if the process dies mid-task?

The three options at a glance

from fastapi import FastAPI, BackgroundTasks
import asyncio
from arq.connections import ArqRedis

app = FastAPI()

@app.post("/option-a")
async def option_a(bt: BackgroundTasks):
    bt.add_task(send_welcome_email, user_id=42)
    return {"status": "queued"}

@app.post("/option-b")
async def option_b():
    asyncio.create_task(send_welcome_email(user_id=42))
    return {"status": "queued"}

@app.post("/option-c")
async def option_c(redis: ArqRedis):
    await redis.enqueue_job("send_welcome_email", user_id=42)
    return {"status": "queued"}

All three return immediately. Only one of them survives a crash.

Option A: FastAPI BackgroundTasks

BackgroundTasks is a Starlette feature exposed through FastAPI's dependency injection. The task runs after the response is returned but before the request is fully closed from Starlette's perspective. The function executes in the same event loop as the request that scheduled it.

What that buys you:

Errors raised in the task are logged via the standard exception middleware
The task is guaranteed to start (Starlette awaits it as part of the response lifecycle)
Zero infrastructure: no Redis, no worker process, no broker

What it costs:

The task blocks the worker from accepting the next request on that connection until it finishes
If the process is killed (SIGTERM during deploy, OOM, segfault from a C extension), the task is gone with no record
No retries, no scheduling, no concurrency control across workers

A 200ms email send is fine. A 30-second PDF generation is not \u2014 uvicorn's default graceful shutdown timeout is 30 seconds, and your worker will get SIGKILLed mid-render during a routine deploy.

Use it when: the task is fast (under 1s), idempotent on the caller side, and "best-effort" semantics are acceptable. Sending a confirmation email after signup is the canonical example, because the user can always click "resend".

Option B: asyncio.create_task

This is what you reach for when you've outgrown BackgroundTasks and don't want to add Redis yet. It schedules a coroutine on the running event loop and returns a Task handle that the request handler does not await.

import asyncio
import logging

log = logging.getLogger(__name__)

async def safe_background(coro):
    try:
        await coro
    except Exception:
        log.exception("background task failed")

@app.post("/index-document")
async def index_document(doc_id: str):
    asyncio.create_task(safe_background(reindex(doc_id)))
    return {"status": "queued"}

Two non-obvious gotchas trip people here:

1. Strong references. asyncio.create_task returns a Task that the event loop holds only weakly in some Python versions. If nothing else keeps a reference, garbage collection can cancel the task before it runs. The Python docs explicitly warn about this:

_BACKGROUND: set[asyncio.Task] = set()

def fire_and_forget(coro):
    task = asyncio.create_task(coro)
    _BACKGROUND.add(task)
    task.add_done_callback(_BACKGROUND.discard)

2. Naked exceptions vanish. A create_task coroutine that raises and is never awaited produces a Task exception was never retrieved warning at GC time, not at failure time. Wrap every fire-and-forget coroutine in a try/except wrapper that logs.

The lifecycle problem is identical to BackgroundTasks: SIGTERM during a deploy kills the task. The advantage over Option A is that you can spawn many tasks per request without blocking the response, and you control the wrapper (timeout, retry-once, structured logging).

A real-world pattern from a polyglot orchestrator I work in: a /replenish-all endpoint dispatches research jobs to 9 downstream agents. Returning HTTP 202 with asyncio.create_task cuts the operator-facing latency from 4 minutes to 200ms, and a separate sweep endpoint reports back when the agents finish. nginx proxy_read_timeout is 370s as a safety net, but the operator never waits.

Use it when: you need fan-out beyond one task per request, the work is bounded (under 10s), losing a task on crash is annoying but recoverable, and you don't yet want a separate worker process.

Option C: Redis-backed queue (arq)

arq is asyncio-native, uses Redis as the broker, and has the smallest API surface of the durable options. RQ and Celery exist; arq fits FastAPI better because it shares the event loop model.

from arq import create_pool
from arq.connections import RedisSettings

async def reindex(ctx, doc_id: str):
    # ctx contains job_id, enqueue_time, score
    await heavy_work(doc_id)

class WorkerSettings:
    functions = [reindex]
    redis_settings = RedisSettings(host="redis", port=6379)
    max_tries = 3
    job_timeout = 300

The handler now takes one line:

@app.post("/index-document")
async def index_document(doc_id: str, redis: ArqRedis = Depends(get_redis)):
    job = await redis.enqueue_job("reindex", doc_id)
    return {"status": "queued", "job_id": job.job_id}

What you get that the in-process options can't deliver:

Crash survival: a job is committed to Redis before the response returns; if the API process dies, the worker picks it up
Retry semantics: max_tries=3 with exponential backoff, dead-letter queue for poison messages
Scheduling: _defer_until and cron-style recurring jobs without a separate scheduler
Failure isolation: a worker that hits an OOM only kills its own job, not the API process serving live requests
Backpressure: the queue grows under load instead of consuming all your event loop slots

What you pay:

A Redis instance (free if you're already running one \u2014 most stacks are)
A separate arq worker process to deploy and monitor
The serialization boundary: arguments must be msgpack-serializable, which means Pydantic models need .model_dump() before enqueue

The 80% rule: anything that takes more than ~5 seconds, anything that touches an external API with retry-worthy failure modes (Stripe webhooks, OpenAI calls, S3 uploads), and anything where "we lost a job" would warrant a Slack message belongs on the queue.

The decision tree

Question	If yes	If no
Will losing this task on crash cause user-visible damage?	Option C (queue)	Continue
Does it take more than ~3 seconds?	Option C (queue)	Continue
Do you need fan-out (many tasks per request)?	Option B (`create_task`)	Continue
Is it a single fast side effect tied to this request?	Option A (`BackgroundTasks`)	Reconsider \u2014 maybe it's not background work

Three specific gotchas worth pinning to the office wall:

Sync functions in BackgroundTasks run in a threadpool. FastAPI inspects the function signature; def send_email(...) runs on a thread, async def send_email(...) runs on the event loop. Mixing the two without realizing it produces inconsistent latency under load.

asyncio.create_task does not survive Depends cleanup. If the coroutine references a database session opened via Depends(get_db), that session closes when the response returns. The task then blows up with a closed connection error two seconds later. Pass primitive arguments, or open a fresh session inside the task.

arq's max_tries includes the first attempt. max_tries=3 means one initial attempt plus two retries, not three retries on top. The naming trips up people coming from Celery.

When to escalate beyond arq

arq covers single-region, single-Redis deployments cleanly. Reach for Celery + Redis/RabbitMQ when you need cross-language workers (Python enqueueing for a Go consumer) or chord/group primitives. Reach for Temporal when "this workflow has 12 steps and step 7 talks to Stripe" is a literal requirement and you want durable execution semantics for free. Don't start there \u2014 the operational burden is real, and arq handles 95% of background work in a single FastAPI service.

The pattern I keep coming back to: in-process for "fire and pray, but pray briefly", queue for "this must happen". The middle ground (create_task with manual retry logic) is fine as a transitional step, but every line of homegrown retry code is a line you'll wish you'd spent setting up arq.

References: