Arc<Mutex>

Guides

Rust walkthroughs

← Back to all guides

How does tower::ServiceBuilder simplify middleware ordering and composition?

tower::ServiceBuilder provides a fluent, type-safe API for composing middleware layers in the correct order, eliminating the error-prone manual nesting that results from directly wrapping services. Without ServiceBuilder, adding multiple middleware layers requires inside-out construction: Middleware3::new(Middleware2::new(Middleware1::new(service))), where the outermost middleware appears innermost in the code. ServiceBuilder inverts this to a natural left-to-right, top-to-bottom reading order: layers are added in execution order, making the code match the mental model of request processing. The builder uses the Layer trait to wrap services, producing a final composed service type at compile time. This approach prevents runtime errors from incorrect ordering, enables conditional middleware inclusion, and makes the middleware stack visually explicit—critical for debugging complex service pipelines where order affects correctness, such as authentication before authorization, or timeout before rate limiting.

The Middleware Nesting Problem

use tower::{Service, ServiceBuilder};
use tower::layer::Layer;
use std::task::{Context, Poll};
use pin_project_lite::pin_project;
use std::pin::Pin;
 
// Imagine we have three middleware layers
// Without ServiceBuilder, composition looks like this:
 
fn manual_nesting_example<S>(service: S) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    // This reads inside-out:
    // - TimeoutMiddleware wraps RateLimitMiddleware
    // - RateLimitMiddleware wraps LoggingMiddleware
    // - LoggingMiddleware wraps service
    // 
    // Execution order: Timeout -> RateLimit -> Logging -> service
    // But the code reads in the opposite order!
    
    // TimeoutMiddleware::new(
    //     RateLimitMiddleware::new(
    //         LoggingMiddleware::new(service)
    //     )
    // )
    
    // This is confusing: the outermost middleware is the innermost in code
    service  // Placeholder
}

Manual nesting reverses the natural reading order of middleware.

ServiceBuilder for Readable Composition

use tower::{ServiceBuilder, Service, ServiceExt};
use tower::layer::Layer;
use std::time::Duration;
 
// Simplified middleware examples for demonstration
struct LoggingLayer;
struct RateLimitLayer;
struct TimeoutLayer {
    duration: Duration,
}
 
impl<S> Layer<S> for LoggingLayer {
    type Service = LoggingService<S>;
    fn layer(&self, inner: S) -> Self::Service {
        LoggingService { inner }
    }
}
 
impl<S> Layer<S> for RateLimitLayer {
    type Service = RateLimitService<S>;
    fn layer(&self, inner: S) -> Self::Service {
        RateLimitService { inner }
    }
}
 
impl<S> Layer<S> for TimeoutLayer {
    type Service = TimeoutService<S>;
    fn layer(&self, inner: S) -> Self::Service {
        TimeoutService { inner, duration: self.duration }
    }
}
 
// Wrapper services
struct LoggingService<S> { inner: S }
struct RateLimitService<S> { inner: S }
struct TimeoutService<S> { 
    inner: S,
    duration: Duration,
}
 
fn service_builder_example<S>(service: S) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    // With ServiceBuilder, the order matches execution order:
    // 1. Timeout (outermost - first to see request, last to see response)
    // 2. RateLimit
    // 3. Logging (innermost - last to see request, first to see response)
    // 4. service
    
    ServiceBuilder::new()
        .layer(TimeoutLayer { duration: Duration::from_secs(30) })
        .layer(RateLimitLayer)
        .layer(LoggingLayer)
        .service(service)
}

ServiceBuilder reads top-to-bottom, matching execution order.

Middleware Execution Order

use tower::{Service, ServiceBuilder};
use tower::layer::Layer;
use std::task::{Context, Poll};
use std::pin::Pin;
 
// A middleware that prints on the way in and out
struct PrintLayer(&'static str);
 
struct PrintService<S> {
    inner: S,
    name: &'static str,
}
 
impl<S, R> Service<R> for PrintService<S>
where
    S: Service<R>,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = S::Future;
 
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        self.inner.poll_ready(cx)
    }
 
    fn call(&mut self, req: R) -> Self::Future {
        println!("-> entering {}", self.name);
        let fut = self.inner.call(req);
        // In real middleware, we'd wrap the future to print on exit
        fut
    }
}
 
impl<S> Layer<S> for PrintLayer {
    type Service = PrintService<S>;
    fn layer(&self, inner: S) -> Self::Service {
        PrintService { inner, name: self.0 }
    }
}
 
// A simple service
struct EchoService;
impl Service<String> for EchoService {
    type Response = String;
    type Error = std::convert::Infallible;
    type Future = std::future::Ready<Result<String, Self::Error>>;
 
    fn poll_ready(&mut self, _: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        Poll::Ready(Ok(()))
    }
 
    fn call(&mut self, req: String) -> Self::Future {
        println!("   [ECHO SERVICE] processing: {}", req);
        std::future::ready(Ok(req))
    }
}
 
fn main() {
    let mut service = ServiceBuilder::new()
        .layer(PrintLayer("A"))
        .layer(PrintLayer("B"))
        .layer(PrintLayer("C"))
        .service(EchoService);
 
    // Execution order:
    // -> entering A (outermost, first)
    // -> entering B
    // -> entering C (innermost, last before service)
    //    [ECHO SERVICE] processing
    // <- exiting C (first to see response)
    // <- exiting B
    // <- exiting A (last to see response)
}

The first layer added is the outermost—it sees requests first and responses last.

Conditional Middleware Inclusion

use tower::{ServiceBuilder, Service};
use tower::layer::Layer;
use std::time::Duration;
 
struct Config {
    enable_logging: bool,
    enable_rate_limiting: bool,
    timeout_secs: Option<u64>,
}
 
fn build_service<S>(service: S, config: &Config) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    // ServiceBuilder allows conditional layer addition
    let mut builder = ServiceBuilder::new();
    
    if let Some(secs) = config.timeout_secs {
        builder = builder.layer(TimeoutLayer { duration: Duration::from_secs(secs) });
    }
    
    if config.enable_rate_limiting {
        builder = builder.layer(RateLimitLayer);
    }
    
    if config.enable_logging {
        builder = builder.layer(LoggingLayer);
    }
    
    builder.service(service)
}

Conditional middleware is straightforward with ServiceBuilder.

Type Inference and the Identity Service

use tower::{ServiceBuilder, Service, identity::Identity};
use tower::layer::Layer;
 
fn identity_example() {
    // ServiceBuilder starts with Identity - a no-op service
    let builder = ServiceBuilder::new();
    
    // Adding layers transforms the service type
    let service = builder
        .layer(LoggingLayer)
        .service(EchoService);
    
    // The final type is a composition of all layers
    // LoggingService<EchoService>
    
    // Without any layers, Identity passes through unchanged
    let identity_service = ServiceBuilder::new()
        .service(Identity::new());
    
    // Identity::new() is a no-op service
}
 
struct EchoService;
impl Service<String> for EchoService {
    type Response = String;
    type Error = std::convert::Infallible;
    type Future = std::future::Ready<Result<String, Self::Error>>;
 
    fn poll_ready(&mut self, _: &mut std::task::Context<'_>) -> Poll<Result<(), Self::Error>> {
        Poll::Ready(Ok(()))
    }
 
    fn call(&mut self, req: String) -> Self::Future {
        std::future::ready(Ok(req))
    }
}
 
struct LoggingLayer;
struct LoggingService<S> { inner: S }
impl<S> Layer<S> for LoggingLayer {
    type Service = LoggingService<S>;
    fn layer(&self, inner: S) -> Self::Service {
        LoggingService { inner }
    }
}

ServiceBuilder uses Identity as the base service type.

Real-World Example with tower-http

use tower::{ServiceBuilder, Service};
use tower_http::{
    trace::TraceLayer,
    timeout::TimeoutLayer,
    limit::RateLimitLayer,
    compression::CompressionLayer,
};
use std::time::Duration;
 
// Example with real tower-http middleware
fn build_http_service<S>(service: S) -> impl Service<http::Request<hyper::body::Body>>
where
    S: Service<http::Request<hyper::body::Body>>,
{
    ServiceBuilder::new()
        // 1. Trace all requests (outermost)
        .layer(TraceLayer::new_for_http())
        
        // 2. Enforce timeout
        .layer(TimeoutLayer::new(Duration::from_secs(30)))
        
        // 3. Rate limit requests
        .layer(RateLimitLayer::new(100, Duration::from_secs(1)))
        
        // 4. Compress responses (innermost, applied last)
        .layer(CompressionLayer::new())
        
        .service(service)
}
 
// The execution flow:
// Request -> Trace -> Timeout -> RateLimit -> Compression -> Service
// Response <- Trace <- Timeout <- RateLimit <- Compression <- Service

Real middleware stacks benefit from readable ordering.

Layer Trait Implementation

use tower::layer::Layer;
use std::time::Duration;
 
// Custom middleware that adds a header
struct AddHeaderLayer {
    name: &'static str,
    value: &'static str,
}
 
struct AddHeaderService<S> {
    inner: S,
    name: &'static str,
    value: &'static str,
}
 
impl<S> Layer<S> for AddHeaderLayer {
    type Service = AddHeaderService<S>;
    
    fn layer(&self, inner: S) -> Self::Service {
        AddHeaderService {
            inner,
            name: self.name,
            value: self.value,
        }
    }
}
 
// The Layer trait enables ServiceBuilder composition
fn custom_layer_example<S>(service: S) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    ServiceBuilder::new()
        .layer(AddHeaderLayer { name: "X-Custom", value: "hello" })
        .layer(AddHeaderLayer { name: "X-Another", value: "world" })
        .service(service)
}

Implementing Layer makes middleware composable with ServiceBuilder.

Cloning Layers for Multiple Services

use tower::{ServiceBuilder, Service};
use tower::layer::Layer;
use std::sync::Arc;
 
fn shared_middleware_example<S>(services: Vec<S>) -> Vec<impl Service<String, Response = String>>
where
    S: Service<String, Response = String>,
{
    // ServiceBuilder can be reused to apply the same middleware stack
    let builder = ServiceBuilder::new()
        .layer(LoggingLayer)
        .layer(RateLimitLayer);
    
    services
        .into_iter()
        .map(|s| builder.clone().service(s))
        .collect()
}
 
// Note: Layers must be Clone for this to work
// Alternatively, use layer_fn for function-based layers
 
#[derive(Clone)]
struct LoggingLayer;
#[derive(Clone)]
struct RateLimitLayer;

A ServiceBuilder can be cloned to apply the same stack to multiple services.

Function-Based Layers

use tower::{ServiceBuilder, layer::layer_fn};
use std::time::Duration;
 
fn function_layer_example<S>(service: S) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    ServiceBuilder::new()
        // Use layer_fn for simple middleware without a dedicated Layer type
        .layer(layer_fn(|inner: S| TimingService { inner }))
        .layer(layer_fn(|inner: S| LoggingService { inner }))
        .service(service)
}
 
struct TimingService<S> { inner: S }
struct LoggingService<S> { inner: S }

layer_fn simplifies creating layers from closure-like constructors.

ServiceBuilder and IntoService

use tower::{ServiceBuilder, Service, ServiceExt};
use tower::layer::Layer;
 
async fn ready_service_example() {
    // ServiceBuilder also has convenience methods
    
    let service = ServiceBuilder::new()
        .layer(LoggingLayer)
        .service(EchoService);
    
    // .service() consumes the builder and creates the service
    // Alternatively, .into_service() is an alias
    
    // Can also use .into_inner() to get just the layered service
    // without consuming the builder (requires IntoService trait)
}
 
struct EchoService;
impl Service<String> for EchoService {
    type Response = String;
    type Error = std::convert::Infallible;
    type Future = std::future::Ready<Result<String, Self::Error>>;
 
    fn poll_ready(&mut self, _: &mut std::task::Context<'_>) -> Poll<Result<(), Self::Error>> {
        Poll::Ready(Ok(()))
    }
 
    fn call(&mut self, req: String) -> Self::Future {
        std::future::ready(Ok(req))
    }
}

ServiceBuilder provides multiple ways to finalize the service.

Error Type Propagation

use tower::{ServiceBuilder, Service, BoxError};
use std::fmt::Debug;
 
// When composing middleware, error types must be compatible
// ServiceBuilder helps manage this through type inference
 
fn error_handling_example<S>(service: S) -> impl Service<String, Response = String, Error = BoxError>
where
    S: Service<String, Response = String>,
    S::Error: Into<BoxError> + 'static,
{
    ServiceBuilder::new()
        .layer(TimeoutLayer { duration: Duration::from_secs(30) })
        .layer(LoggingLayer)
        .service(service)
}
 
// The service method expects the inner service's error to be compatible
// with the error types produced by the middleware layers

Error types must be compatible across the middleware stack.

Comparison: Manual vs ServiceBuilder

use tower::{ServiceBuilder, Service};
use tower::layer::Layer;
use std::time::Duration;
 
struct TimeoutLayer { duration: Duration }
struct RateLimitLayer;
struct LoggingLayer;
struct AuthLayer;
 
// Manual nesting (inside-out, confusing):
fn manual_stack<S>(service: S) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    // The order here is confusing:
    // - Auth is outermost (executed first)
    // - Logging is next
    // - RateLimit is next
    // - Timeout is innermost (executed last before service)
    // But the code reads: Timeout wraps RateLimit wraps Logging wraps Auth wraps service
    
    TimeoutLayer { duration: Duration::from_secs(30) }
        .layer(
            RateLimitLayer.layer(
                LoggingLayer.layer(
                    AuthLayer.layer(service)
                )
            )
        )
}
 
// ServiceBuilder (natural order, clear):
fn builder_stack<S>(service: S) -> impl Service<String, Response = String>
where
    S: Service<String, Response = String>,
{
    ServiceBuilder::new()
        // Execution order matches reading order:
        .layer(AuthLayer)           // 1. Authenticate (first)
        .layer(LoggingLayer)         // 2. Log request
        .layer(RateLimitLayer)       // 3. Check rate limits
        .layer(TimeoutLayer { duration: Duration::from_secs(30) })  // 4. Enforce timeout
        .service(service)            // 5. Handle request
}

ServiceBuilder makes the middleware order explicit and readable.

Stack Abstraction for Reusable Middleware Collections

use tower::{ServiceBuilder, Service};
use tower::layer::Layer;
 
// Define a reusable middleware stack
struct ProductionStack {
    timeout_secs: u64,
    rate_limit: usize,
}
 
impl ProductionStack {
    fn apply<S>(&self, service: S) -> impl Service<String, Response = String>
    where
        S: Service<String, Response = String>,
    {
        ServiceBuilder::new()
            .layer(TimeoutLayer { duration: Duration::from_secs(self.timeout_secs) })
            .layer(RateLimitLayer)
            .layer(LoggingLayer)
            .service(service)
    }
}
 
// Define another stack for development
struct DevelopmentStack;
 
impl DevelopmentStack {
    fn apply<S>(&self, service: S) -> impl Service<String, Response = String>
    where
        S: Service<String, Response = String>,
    {
        ServiceBuilder::new()
            .layer(LoggingLayer)  // Only logging in development
            .service(service)
    }
}
 
fn stack_example<S>(service: S, production: bool) -> Box<dyn Service<String, Response = String>>
where
    S: Service<String, Response = String> + 'static,
{
    if production {
        Box::new(ProductionStack { timeout_secs: 30, rate_limit: 100 }.apply(service))
    } else {
        Box::new(DevelopmentStack.apply(service))
    }
}

ServiceBuilder enables reusable middleware stack definitions.

The ServiceBuilder Type Signature

use tower::ServiceBuilder;
 
// ServiceBuilder is parameterized by the accumulated layers
// The type grows with each layer added
 
// Initial: ServiceBuilder<Identity>
let builder = ServiceBuilder::new();
 
// After one layer: ServiceBuilder<Layer1<Identity>>
let builder = builder.layer(LoggingLayer);
 
// After two layers: ServiceBuilder<Layer2<Layer1<Identity>>>
let builder = builder.layer(RateLimitLayer);
 
// The type encodes the entire middleware stack at compile time
// This ensures type safety and enables inlining/optimization

Each layer() call adds to the compile-time type.

Synthesis

Ordering comparison:

Approach Reading Direction Execution Direction Complexity
Manual nesting Inside-out Outside-in O(n²) mental
ServiceBuilder Top-to-bottom Top-to-bottom O(n) mental

ServiceBuilder capabilities:

ServiceBuilder::new()
    // Methods available:
    .layer(L)           // Add a layer
    .layer_fn(f)        // Add a function-based layer
    .service(s)         // Finalize with a service
    .into_inner()       // Get the built service
    .clone()            // Clone for reuse (if layers are Clone)
    .into_service(s)    // Alias for .service()
    .map_request(f)     // Transform requests
    .map_response(f)    // Transform responses
    .map_err(f)         // Transform errors

Key insight: ServiceBuilder solves a fundamental ergonomic problem in middleware composition: the disconnect between code structure and execution order. Traditional middleware composition uses decorator pattern nesting, where Outer(Inner(Service)) means Outer wraps Inner which wraps Service. This means the outermost middleware—the first to handle requests—appears innermost in the code, creating a cognitive burden that scales quadratically with the number of layers. ServiceBuilder uses the builder pattern with Layer trait composition to invert this: you write layers in request-processing order, and the builder handles the nesting internally. The type system ensures correctness—the final service type is a composition of all layer types, enabling inlining and eliminating runtime dispatch. This matters because middleware order is often semantically significant: authentication must precede authorization, rate limiting should precede expensive operations, and timeouts should wrap operations that might hang. Getting this order wrong causes subtle bugs that ServiceBuilder's explicit ordering makes visible and auditable. The builder also enables patterns that are awkward with manual nesting: conditional middleware inclusion, reusable stacks, and cloning for multiple services. The Layer trait abstraction means any middleware implementing it becomes composable, and layer_fn provides a zero-boilerplate way to wrap simple transformations.