Arc<Mutex>

Guides

Rust walkthroughs

ā† Back to all guides

What is the purpose of tower::ServiceBuilder for composing multiple middleware layers?

tower::ServiceBuilder solves the middleware composition problem by providing a fluent builder pattern that stacks multiple Layer implementations into a single composed layer, eliminating the deeply nested generic types that arise from manually wrapping servicesā€”instead of Middleware3<M, Middleware2<M, Middleware1<M, S>>>, you get clean sequential syntax with full type inference. The builder collects layers and applies them in order, managing the type complexity internally.

The Middleware Nesting Problem

use tower::{Service, Layer};
use tower::limit::concurrency::ConcurrencyLimitLayer;
use tower::timeout::TimeoutLayer;
use tower::retry::RetryLayer;
use std::time::Duration;
 
// Without ServiceBuilder, middleware nesting creates complex types
fn manual_composition<S>(service: S) -> impl tower::Service<String>
where
    S: Service<String> + Clone + Send + 'static,
    S::Future: Send,
{
    // Each layer wraps the previous, creating nested types
    let layered = TimeoutLayer::new(Duration::from_secs(30))
        .layer(service);
    
    let layered = ConcurrencyLimitLayer::new(100)
        .layer(layered);
    
    let layered = RetryLayer::new()
        .layer(layered);
    
    // Return type would be:
    // Retry<ConcurrencyLimit<Timeout<S>>>
    
    layered
}

Each manual .layer() call adds another wrapper type, making type signatures unwieldy.

Basic ServiceBuilder Usage

use tower::ServiceBuilder;
use tower::limit::concurrency::ConcurrencyLimitLayer;
use tower::timeout::TimeoutLayer;
use std::time::Duration;
 
fn service_builder_basic<S>(service: S) -> impl tower::Service<String>
where
    S: Service<String> + Clone + Send + 'static,
    S::Future: Send,
{
    // ServiceBuilder collects layers in order
    let layered = ServiceBuilder::new()
        .layer(TimeoutLayer::new(Duration::from_secs(30)))
        .layer(ConcurrencyLimitLayer::new(100))
        .service(service);
    
    // Returns composed service with all layers applied
    // Clean syntax, types handled internally
    
    layered
}

ServiceBuilder provides a fluent API where .layer() adds middleware and .service() applies all layers.

Layer Order and Execution

use tower::ServiceBuilder;
use tower::{Service, Layer};
use std::task::{Context, Poll};
use std::future::Future;
use std::pin::Pin;
 
// Custom layer for demonstration
struct LoggingLayer(&'static str);
impl<S> Layer<S> for LoggingLayer {
    type Service = LoggingService<S>;
    fn layer(&self, inner: S) -> Self::Service {
        LoggingService(inner, self.0)
    }
}
 
struct LoggingService<S>(&'static str, S);
impl<S, Request> Service<Request> for LoggingService<S>
where
    S: Service<Request>,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = S::Future;
    
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        println!("poll_ready: {}", self.0);
        self.1.poll_ready(cx)
    }
    
    fn call(&mut self, req: Request) -> Self::Future {
        println!("call: {}", self.0);
        self.1.call(req)
    }
}
 
fn layer_order() {
    // Layer order matters!
    // Layers are applied in the order specified
    // Execution order: outer to inner on call, inner to outer on response
    
    let builder = ServiceBuilder::new()
        .layer(LoggingLayer("first"))   // Outer layer
        .layer(LoggingLayer("second"))  // Middle layer
        .layer(LoggingLayer("third"));  // Inner layer (closest to service)
    
    // When a request comes in:
    // 1. First (outer) sees it first
    // 2. Second sees it next
    // 3. Third (inner) sees it last
    // 4. Service processes
    // 5. Third returns response
    // 6. Second returns response
    // 7. First returns response
}

Layers are applied in order; requests flow through outer to inner, responses flow inner to outer.

Comparing Manual vs Builder Composition

use tower::{ServiceBuilder, Service, Layer};
use tower::limit::concurrency::ConcurrencyLimitLayer;
use tower::timeout::TimeoutLayer;
use tower::buffer::BufferLayer;
use std::time::Duration;
 
fn manual_vs_builder() {
    // Manual composition - nested generic types
    fn manual<S>(service: S) -> impl Service<String>
    where
        S: Service<String> + Clone + Send + 'static,
    {
        let s1 = TimeoutLayer::new(Duration::from_secs(10))
            .layer(service);
        let s2 = ConcurrencyLimitLayer::new(50)
            .layer(s1);
        let s3 = BufferLayer::new(100)
            .layer(s2);
        s3
        // Type: Buffer<ConcurrencyLimit<Timeout<S>>>
    }
    
    // ServiceBuilder - clean composition
    fn with_builder<S>(service: S) -> impl Service<String>
    where
        S: Service<String> + Clone + Send + 'static,
    {
        ServiceBuilder::new()
            .layer(TimeoutLayer::new(Duration::from_secs(10)))
            .layer(ConcurrencyLimitLayer::new(50))
            .layer(BufferLayer::new(100))
            .service(service)
        // Type: Service<Timeout, ConcurrencyLimit, Buffer, S> (conceptual)
    }
    
    // Benefits of ServiceBuilder:
    // 1. Readable order (first layer = outermost)
    // 2. Easy to add/remove layers
    // 3. Type inference works well
    // 4. Can reuse builder for multiple services
}

ServiceBuilder produces cleaner code with equivalent functionality.

Reusing ServiceBuilder

use tower::ServiceBuilder;
use tower::limit::concurrency::ConcurrencyLimitLayer;
use tower::timeout::TimeoutLayer;
use std::time::Duration;
 
fn reuse_builder() {
    // Create builder once, apply to multiple services
    let builder = ServiceBuilder::new()
        .layer(TimeoutLayer::new(Duration::from_secs(30)))
        .layer(ConcurrencyLimitLayer::new(100));
    
    // Apply to different services
    let service_a = builder.service(make_service_a());
    let service_b = builder.service(make_service_b());
    let service_c = builder.service(make_service_c());
    
    // All three services have the same middleware stack
    
    // builder is consumed after .service(), but we can clone:
    fn multiple_services<S>(services: Vec<S>) -> Vec<impl tower::Service<String>>
    where
        S: Service<String> + Clone + Send + 'static,
    {
        let builder = ServiceBuilder::new()
            .layer(TimeoutLayer::new(Duration::from_secs(30)));
        
        // Clone builder for each service
        services.into_iter()
            .map(|s| builder.clone().service(s))
            .collect()
    }
}

Create a builder once and apply it to multiple services for consistent middleware configuration.

Common Middleware Patterns

use tower::ServiceBuilder;
use tower::limit::{concurrency::ConcurrencyLimitLayer, rate::RateLimitLayer};
use tower::timeout::TimeoutLayer;
use tower::retry::{RetryLayer, backoff::ExponentialBackoff};
use tower::buffer::BufferLayer;
use std::time::Duration;
 
fn common_patterns() {
    // Production HTTP service pattern
    let builder = ServiceBuilder::new()
        // Request timeout
        .layer(TimeoutLayer::new(Duration::from_secs(30)))
        // Rate limiting (requests per second)
        .layer(RateLimitLayer::new(100, Duration::from_secs(1)))
        // Concurrency limit (in-flight requests)
        .layer(ConcurrencyLimitLayer::new(1000))
        // Buffer for backpressure
        .layer(BufferLayer::new(100));
    
    // Retry with backoff pattern
    let retry_builder = ServiceBuilder::new()
        .layer(RetryLayer::new(ExponentialBackoff::new(
            Duration::from_millis(100),
            Duration::from_secs(10),
        )))
        .layer(TimeoutLayer::new(Duration::from_secs(30)));
    
    // All patterns are readable and maintainable
}

Common production patterns are easy to express with ServiceBuilder.

The Layer Trait

use tower::Layer;
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
 
// Layers wrap services to add behavior
trait Layer<S> {
    type Service;
    fn layer(&self, inner: S) -> Self::Service;
}
 
// Example: A simple logging layer
struct LoggingLayer;
 
impl<S> Layer<S> for LoggingLayer {
    type Service = LoggingService<S>;
    
    fn layer(&self, inner: S) -> Self::Service {
        LoggingService(inner)
    }
}
 
struct LoggingService<S>(S);
 
impl<S, Request> tower::Service<Request> for LoggingService<S>
where
    S: tower::Service<Request>,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = S::Future;
    
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        self.0.poll_ready(cx)
    }
    
    fn call(&mut self, req: Request) -> Self::Future {
        println!("Received request");
        self.0.call(req)
    }
}
 
// Layers are composable because they transform Service -> Service
// ServiceBuilder just chains these transformations

Each Layer<S> transforms a Service into a new Service with added behavior.

The Service Trait

use tower::Service;
use std::future::Future;
use std::task::{Context, Poll};
 
// Service is the core abstraction
trait Service<Request> {
    type Response;
    type Error;
    type Future: Future<Output = Result<Self::Response, Self::Error>>;
    
    // Check if service is ready
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>;
    
    // Process the request
    fn call(&mut self, req: Request) -> Self::Future;
}
 
// Middleware layers implement Layer<S> to wrap Service<S>
// Each layer adds behavior to poll_ready and/or call
// ServiceBuilder composes multiple layers efficiently
 
// The key insight: ServiceBuilder manages the composition
// Each layer wraps the previous, building the middleware stack

Service represents an async function that takes a request and returns a response.

Conditional Layer Application

use tower::ServiceBuilder;
use tower::timeout::TimeoutLayer;
use tower::limit::concurrency::ConcurrencyLimitLayer;
use std::time::Duration;
 
fn conditional_layers(enabled: bool) {
    let builder = ServiceBuilder::new();
    
    // ServiceBuilder supports conditional layering via into_inner
    let builder = if enabled {
        builder.layer(TimeoutLayer::new(Duration::from_secs(30)))
    } else {
        builder
    };
    
    // Or use option_layer pattern
    fn optional_timeout(enable: bool) -> Option<TimeoutLayer> {
        if enable {
            Some(TimeoutLayer::new(Duration::from_secs(30)))
        } {
            None
        }
    }
    
    // Tower provides identity layer for conditional composition
    use tower::util::Identity;
    
    let builder = ServiceBuilder::new()
        .layer(
            if enabled {
                TimeoutLayer::new(Duration::from_secs(30)).into_inner()
            } else {
                Identity::new()
            }
        );
}

Conditional layers can be applied using builder methods or Identity as a no-op layer.

Integration with tower-http

use tower::ServiceBuilder;
use tower_http::{
    trace::TraceLayer,
    compression::CompressionLayer,
    cors::CorsLayer,
    limit::RequestBodyLimitLayer,
    classify::ServerErrorsFailureClass,
};
use std::time::Duration;
 
fn tower_http_integration() {
    // tower-http provides many useful layers
    let builder = ServiceBuilder::new()
        // Tracing/logging
        .layer(TraceLayer::new_for_http())
        // CORS handling
        .layer(CorsLayer::permissive())
        // Request body size limit
        .layer(RequestBodyLimitLayer::new(10 * 1024 * 1024))  // 10MB
        // Response compression
        .layer(CompressionLayer::new());
    
    // Apply to hyper or axum service
    // let service = builder.service(my_service);
}

tower-http layers integrate seamlessly with ServiceBuilder.

Integration with Tonic (gRPC)

use tower::ServiceBuilder;
use tower::timeout::TimeoutLayer;
use tower::limit::concurrency::ConcurrencyLimitLayer;
use tonic::transport::Server;
use std::time::Duration;
 
fn tonic_integration() {
    // ServiceBuilder works with Tonic gRPC services
    let middleware = ServiceBuilder::new()
        .layer(TimeoutLayer::new(Duration::from_secs(30)))
        .layer(ConcurrencyLimitLayer::new(100));
    
    // let service = MyGrpcService::new();
    // let service = middleware.service(service);
    
    // Server::builder()
    //     .add_service(service)
    //     .serve(addr)
    //     .await;
}

Tonic gRPC services can be wrapped with ServiceBuilder middleware.

Type Inference Benefits

use tower::{ServiceBuilder, Service, Layer};
use tower::timeout::TimeoutLayer;
use tower::limit::concurrency::ConcurrencyLimitLayer;
use std::time::Duration;
 
fn type_inference() {
    // Without ServiceBuilder, types get complex
    fn without_builder<S>(service: S) -> impl Service<String>
    where
        S: Service<String> + Clone + Send + 'static,
    {
        TimeoutLayer::new(Duration::from_secs(10))
            .layer(
                ConcurrencyLimitLayer::new(50)
                    .layer(service)
            )
    }
    // Return type: Timeout<ConcurrencyLimit<S>>
    // Each new layer adds nesting
    
    // With ServiceBuilder, type inference handles it
    fn with_builder<S>(service: S) -> impl Service<String>
    where
        S: Service<String> + Clone + Send + 'static,
    {
        ServiceBuilder::new()
            .layer(TimeoutLayer::new(Duration::from_secs(10)))
            .layer(ConcurrencyLimitLayer::new(50))
            .service(service)
    }
    // impl Service<String> - clean return type
    
    // ServiceBuilder's internal type is complex:
    // Stack<(TimeoutLayer, (ConcurrencyLimitLayer, ()))>
    // But we don't need to write it out
}

ServiceBuilder hides the nested type complexity behind impl Service<Request>.

Stack-based Composition

use tower::ServiceBuilder;
 
fn stack_internals() {
    // ServiceBuilder uses a stack internally
    // Each .layer() pushes onto the stack
    // .service() pops and applies in order
    
    let builder = ServiceBuilder::new();
    // Stack: ()
    
    let builder = builder.layer(tower::timeout::TimeoutLayer::new(std::time::Duration::from_secs(10)));
    // Stack: (TimeoutLayer, ())
    
    let builder = builder.layer(tower::limit::concurrency::ConcurrencyLimitLayer::new(50));
    // Stack: (ConcurrencyLimitLayer, (TimeoutLayer, ()))
    
    // When .service() is called:
    // 1. Pop ConcurrencyLimitLayer, apply to service
    // 2. Pop TimeoutLayer, apply to result
    // Result: Timeout<ConcurrencyLimit<S>>
    
    // The stack ensures layers are applied in the order specified
}

Internally, ServiceBuilder uses a tuple-based stack to track layers.

Service and ServiceMut

use tower::ServiceBuilder;
use tower::{Service, Layer};
 
fn service_vs_service_mut() {
    // ServiceBuilder has two terminal methods:
    
    // .service(inner) - consumes builder, returns composed service
    // .service_mut(&mut inner) - borrows service, returns composed service
    
    // .service() is most common - takes ownership
    fn with_ownership<S>(builder: ServiceBuilder<()>, inner: S) -> impl Service<String>
    where
        S: Service<String>,
    {
        builder.service(inner)
    }
    
    // .service_mut() allows reusing the inner service
    fn with_borrow<S>(builder: ServiceBuilder<()>, inner: &mut S) -> impl Service<String> + '_
    where
        S: Service<String>,
    {
        builder.service(inner)
    }
    
    // Note: service_mut returns a service tied to the borrow
}

.service() takes ownership; .service_mut() borrows the inner service.

Layer Methods

use tower::ServiceBuilder;
use tower::timeout::TimeoutLayer;
use std::time::Duration;
 
fn layer_methods() {
    // .layer(L) - Add a layer
    let builder = ServiceBuilder::new()
        .layer(TimeoutLayer::new(Duration::from_secs(10)));
    
    // .layer_fn(f) - Add a layer from a closure
    let builder = ServiceBuilder::new()
        .layer_fn(|inner| LoggingService(inner));
    
    // .into_inner() - Get the current service (for conditional layers)
    // .check() - Validate the builder (rarely needed)
    
    // Layers can be added conditionally
    fn conditional_timeout(enable: bool) -> ServiceBuilder<()> {
        let builder = ServiceBuilder::new();
        if enable {
            builder.layer(TimeoutLayer::new(Duration::from_secs(10)))
        } else {
            builder
        }
    }
}

ServiceBuilder provides multiple ways to add layers, including from closures.

Error Propagation Through Layers

use tower::ServiceBuilder;
use tower::{Service, Layer};
use tower::timeout::TimeoutLayer;
use std::time::Duration;
 
fn error_propagation() {
    // Each layer can transform errors
    // Timeout layer adds timeout errors
    // Rate limit layer adds rate limit errors
    
    // When composing layers, error types must be compatible
    // Often use Box<dyn Error> or a custom error enum
    
    #[derive(Debug)]
    enum AppError {
        Timeout,
        RateLimit,
        Service(String),
    }
    
    // impl From<TimeoutError> for AppError { ... }
    // impl From<RateLimitError> for AppError { ... }
    
    // Layer composition propagates errors up the stack
    // Outer layers see errors from inner layers
}

Each layer can introduce its own error types; composition requires error type compatibility.

Practical Example: HTTP API Middleware

use tower::ServiceBuilder;
use tower::timeout::TimeoutLayer;
use tower::limit::{concurrency::ConcurrencyLimitLayer, rate::RateLimitLayer};
use tower::buffer::BufferLayer;
use std::time::Duration;
 
fn production_http_api() {
    // Production-ready HTTP API middleware stack
    let builder = ServiceBuilder::new()
        // Global request timeout
        .layer(TimeoutLayer::new(Duration::from_secs(30)))
        
        // Rate limit: 100 requests per second
        .layer(RateLimitLayer::new(100, Duration::from_secs(1)))
        
        // Concurrency limit: 1000 in-flight requests
        .layer(ConcurrencyLimitLayer::new(1000))
        
        // Buffer for backpressure handling
        .layer(BufferLayer::new(100));
    
    // Apply to service
    // let service = builder.service(my_api_service);
    
    // Requests flow:
    // 1. Rate limit check
    // 2. Concurrency limit check  
    // 3. Buffer (if limits exceeded)
    // 4. Timeout enforcement
    // 5. Service handler
    
    // Benefits:
    // - Prevents resource exhaustion
    // - Graceful degradation under load
    // - Predictable resource usage
    // - Clean separation of concerns
}

A production-ready middleware stack combines multiple layers for resilience and resource management.

Synthesis

ServiceBuilder purpose:

Problem Without ServiceBuilder With ServiceBuilder
Type complexity Nested generics A<B<C<S>>> Clean impl Service<Req>
Readability Inside-out nesting Sequential order
Reusability Manual composition Builder cloning
Add/remove layers Edit complex types Add/remove .layer()
Conditional layers Complex match expressions Conditional builder

Layer order semantics:

// Layers are listed outer-to-inner
ServiceBuilder::new()
    .layer(OuterLayer)    // Applied first, sees request first
    .layer(MiddleLayer)   // Applied second
    .layer(InnerLayer)    // Applied last, closest to service
    .service(inner_service);
 
// Request flow: Outer -> Middle -> Inner -> Service
// Response flow: Service -> Inner -> Middle -> Outer

Key insight: ServiceBuilder exists because manually composing middleware layers creates unwieldy nested type signatures that become impractical as the number of layers grows. By collecting layers into a stack and applying them in sequence, ServiceBuilder lets you write layer().layer().layer().service() instead of Layer3::new(Layer2::new(Layer1::new(service))). The type complexity doesn't disappearā€”it's just hidden behind the builder's internal Stack<L, S> representation. The fluent API makes middleware stacks readable and maintainable, encouraging the pattern of "build a middleware configuration once, apply to many services" that's essential for consistent behavior across an application. The order of layers is crucial: the first layer you add wraps all subsequent layers, making it the outermost middleware that sees requests first and responses last.