How to set workers

[Dominux]

Hi, I was researching how to set up the workers logic, like in gunicorn/uvicorn/etc. But I can't find any ways to serve requests concurrently. Here's a simple example of the app that simulates some async operation:

use std::net::{IpAddr, Ipv4Addr, SocketAddr};

use axum::{routing::get, Router};
use tokio::time;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

const ADDR: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), 8080);

#[tokio::main]
async fn main() {
    tracing_subscriber::registry()
        .with(
            tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| {
                "my_app=debug,tower_http=debug,axum::rejection=trace".into()
            }),
        )
        .with(tracing_subscriber::fmt::layer())
        .init();

    // build our application with a single route
    let app = Router::new().route(
        "/",
        get(|| async move {
            tracing::debug!("got request");
            let _ = time::sleep(time::Duration::from_secs(5)).await;
            tracing::debug!("awaiting done");
            "lol"
        }),
    );

    // run it
    axum::Server::bind(&ADDR)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

After sending 2 requests at the same time it handles them one by one but not at the same time, so it takes 5 * 2 = 10 seconds on it:

2023-10-20T14:53:49.787721Z DEBUG my_app: got request
2023-10-20T14:53:54.792423Z DEBUG my_app: awaiting done
2023-10-20T14:53:54.793303Z DEBUG my_app: got request
2023-10-20T14:53:59.794561Z DEBUG my_app: awaiting done

So I can't get the idea on how to do it with using tower or tower-http

[davidpdrsn]

axum doesn’t handle that. It just does request -> response. Tokio handles the async runtime. You can configure it via the #[tokio::main] macro https://docs.rs/tokio/latest/tokio/attr.main.html

[Dominux]

But how to make the server to listen only a single port and run it in multiple threads?

[davidpdrsn]

how to make the server to listen only a single port

The code you posted already does that. It listens only to 8080.

and run it in multiple threads

This is explained in the docs I linked. Tokio uses a multit threaded runtime by default. So the code you posted uses multiple threads.

[Dominux]

But as we see in tests await blocks taking another request to work with so it's not how concurrently serving requests work. It's not how workers work. It's like it needs something like multiplexing

[davidpdrsn]

There must be something else wrong with that you’re doing. Tokio is multi threaded and hyper+axum does make use of that.

[jplatte]

The code looks perfectly fine. How are you testing it?

[Dominux]

by executing and making requests by opening 2 tabs in the browser

[Dominux]

Oh, sheesh, tried to use curl instead of dumb browser and I got it working this time:

2023-10-21T13:08:10.468567Z DEBUG my_app: got request
2023-10-21T13:08:10.884803Z DEBUG my_app: got request
2023-10-21T13:08:15.469477Z DEBUG my_app: awaiting done
2023-10-21T13:08:15.885457Z DEBUG my_app: awaiting done

I guess browser uses some kinda synchronization between multiple tabs of the single domain name, maybe that's why it works not as expected. Anyway thanks for implying to use other ways to test it.

P.s.: just sick of reading all related questions where the author uses std::thread::sleep so I haven't found any answered ones with proper sleeping strategy