FastApi sync def vs async def ?

In case you are using Fastapi, when does one use def and async def and how does using one make a difference over the other?

Assuming that you scanned this and know a bit of the async stuff.

async

@route(“/users/info”)
async def userInfo():
    user = await fechUser() #(some http request to get users info)
    return process(user)

As we used async, fastapi will run the function for each request on the same single event thread.

As there is an await for a response, fastapi will use this time for serving the other requests. And those requests also wait for response from the external service and hence all of them are served concurrently.

So in this case as most of the time, a request to the route waits for a response from some external service and no single request is taking a long time doing something on the single thread they are running, its fine to use async as fastapi can serve other requests which come at the same time while some of them are waiting.

When we use await how fastapi puts the request in a backlog and serves another request while the response for the first request comes back from the external service depends on the framework implementation.

async with blocking code

@route(“/heavyTask”)
async def userInfo():
    time.sleep(10)

Similar to the above, all the requests use the same event thread for the function execution.But in this case, the code is sleeping for 100 seconds(or consider doing some heavy maths calculation).

Now until the first request completes (10 seconds), the other requests will be waiting and they will be served only when the first request is done.Not only for this route but any request to any other route also gets blocked.Why ? Because the event loop ie. the single thread which executes async functions and also schedules execution of all the requests is blocked on sleep.

Even though you used async, as your code has a blocking operation, all the requests that come after the current served request wait in queue and get blocked.Oops!

So just be sure that any operation down the line is not blocking.

sync

@route(“/heavyTask”)
def userInfo():
    time.sleep(10)

Now similar to above, here we have just changed from async to sync and hence the function userInfo for each request is executed in a separate thread pool and hence one of them doesn’t block the other.

Let's consider the below code

import time
from fastapi import FastAPI

app = FastAPI()

@app.get("/async/sleep")
async def async_sleep():
    time.sleep(100)
    return {"message": "slept for 100 secs"}

@app.get("/sync/sleep")
def sync_sleep():
    time.sleep(100)
    return {"message": "slept for 100 secs"}

@app.get("/async/hello")
async def async_hello():
    return {"message": "hello"}

@app.get("/sync/hello")
def sync_hello():
    return {"message": "hello"}

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

Test 1:

Requests:

1. http://localhost:8000/sync/sleep

2. http://localhost:8000/async/hello

3. http://localhost:8000/sync/hello

Access Log:

INFO: 172.17.0.1:45992 — “GET /async/hello HTTP/1.1” 200 OK
INFO: 172.17.0.1:45992 — “GET /sync/hello HTTP/1.1” 200 OK
INFO: 172.17.0.1:45968 — “GET /sync/sleep HTTP/1.1” 200 OK

Summary:

As you see, we have launched 1, 2, and 3 requests one after the other. The 1st request is for a route which sleeps for 10 seconds.

From the logs, you can see that we get responses for 2 and 3 instantly. We get a response for 1 after 10 seconds. (the logs don’t capture the timestamps and hope you get what I meant when you test it)

As we use def for the sync/sleep route, the request is run on a separate thread pool and other routes are not impacted.

Test 2:

Requests:

1. http://localhost:8000/async/sleep

2. http://localhost:8000/async/hello

3. http://localhost:8000/sync/hello

Access Log:

INFO: 172.17.0.1:46064 — “GET /async/sleep HTTP/1.1” 200 OK
INFO: 172.17.0.1:46066 — “GET /async/hello HTTP/1.1” 200 OK
INFO: 172.17.0.1:46116 — “GET /sync/hello HTTP/1.1” 200 OK

Summary:

The 1st request is for a route which sleeps for 10 seconds but we have used async for it.

From the logs, you can see that only once the 1st request is completed, the 2nd and 3rd are served and processed.

As we used async def for the async/sleep route, the request is run on the same event loop and hence blocks all the other async operations.

Even the 3rd request i.e. /sync/hello is also blocked because the event loop which schedules the execution of requests on the thread pool is blocked by the async sleep function!

So haha, there is no one to schedule the 3rd request on the thread pool though it does not run on the event loop thread.

Now, there are still like which I did not yet come to terms with.

1. How does sync + async work

2. What about def mutating the global state, does it create race conditions

3. What does this mean from here. The initial trigger point is a request itself and from which the route operation gets invoked and which might go down the route with one function calling another etc. Like we don’t explicitly invoke a utility function right?

Any other utility function that you call directly can be created with normal def or async def and FastAPI won't affect the way you call it.

This is in contrast to the functions that FastAPI calls for you: path operation functions and dependencies.

If your utility function is a normal function with def, it will be called directly (as you write it in your code), not in a threadpool, if the function is created with async def then you should await for that function when you call it in your code.

https://github.com/tiangolo/fastapi/issues/2619#issuecomment-762495981
github.com/tiangolo/fastapi/issues/603
news.ycombinator.com/item?id=25992078
gist.github.com/crackerplace/c853fe41d66045..