Arc<Mutex>

Guides

Rust walkthroughs

← Back to all guides

What are the trade-offs between tower::Layer and ServiceBuilder for composing middleware layers?

Layer is the core trait that transforms one service into another, while ServiceBuilder provides a fluent API for composing multiple layers in a specific order with type inference. The trade-off is between Layer's explicit control and composability versus ServiceBuilder's ergonomic chaining that handles the complex nested types automatically.

The Layer Trait

use tower::Layer;
use tower::Service;
 
// Layer is the core trait for middleware transformation
pub trait Layer<Service> {
    type Service;
    
    fn layer(&self, inner: Service) -> Self::Service;
}
 
// The layer method transforms a service
// Input: some service
// Output: a new service that wraps the input

Layer defines the fundamental middleware pattern: wrap a service to add behavior.

Basic Layer Implementation

use tower::{Layer, Service};
use std::task::{Context, Poll};
use std::pin::Pin;
use std::future::Future;
 
// A simple logging layer
pub struct LoggingLayer;
 
impl<S> Layer<S> for LoggingLayer {
    type Service = LoggingService<S>;
    
    fn layer(&self, inner: S) -> Self::Service {
        LoggingService { inner }
    }
}
 
pub struct LoggingService<S> {
    inner: S,
}
 
impl<S, Request> Service<Request> for LoggingService<S>
where
    S: Service<Request>,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = LoggingFuture<S::Future>;
    
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        self.inner.poll_ready(cx)
    }
    
    fn call(&mut self, request: Request) -> Self::Future {
        println!("Handling request");
        LoggingFuture {
            inner: self.inner.call(request),
        }
    }
}
 
pub struct LoggingFuture<F> {
    inner: F,
}
 
impl<F, T, E> Future for LoggingFuture<F>
where
    F: Future<Output = Result<T, E>>,
{
    type Output = Result<T, E>;
    
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let result = self.inner.poll(cx);
        if let Poll::Ready(_) = &result {
            println!("Request completed");
        }
        result
    }
}

A Layer implementation wraps a service and adds behavior to its calls.

Composing Layers Manually

use tower::Layer;
 
fn manual_layer_composition() {
    // Suppose we have layers: Timeout, RateLimit, Logging
    // We want: Logging -> RateLimit -> Timeout -> Service
    
    // Manual composition requires nesting:
    let service = MyService::new();
    
    // Apply Timeout layer
    let timeout_layer = TimeoutLayer::new(Duration::from_secs(30));
    let service = timeout_layer.layer(service);
    
    // Apply RateLimit layer
    let rate_limit_layer = RateLimitLayer::new(100);
    let service = rate_limit_layer.layer(service);
    
    // Apply Logging layer
    let logging_layer = LoggingLayer;
    let service = logging_layer.layer(service);
    
    // Result type is:
    // LoggingService<RateLimitService<TimeoutService<MyService>>>
    // The type becomes increasingly complex with more layers
}

Manual layer composition works but produces complex nested types.

ServiceBuilder for Fluent Composition

use tower::ServiceBuilder;
use std::time::Duration;
 
fn service_builder_composition() {
    // ServiceBuilder provides a fluent API
    // Same result, but cleaner syntax
    
    let service = ServiceBuilder::new()
        .timeout(Duration::from_secs(30))
        .rate_limit(100, Duration::from_secs(1))
        .layer(LoggingLayer)
        .service(MyService::new());
    
    // The order matters: innermost service is applied first
    // Request flow: Logging -> RateLimit -> Timeout -> MyService
    
    // ServiceBuilder handles all the nested types internally
}

ServiceBuilder chains layers fluently and handles type complexity.

Layer Ordering Semantics

use tower::ServiceBuilder;
 
fn layer_ordering() {
    // Order matters: layers are applied in the order listed
    // The last layer listed is the outermost (called first)
    
    // This:
    let service1 = ServiceBuilder::new()
        .layer(LayerA)
        .layer(LayerB)
        .layer(LayerC)
        .service(Inner);
    
    // Is equivalent to:
    let layer_a = LayerA;
    let layer_b = LayerB;
    let layer_c = LayerC;
    let inner = Inner;
    
    let service2 = layer_c.layer(
        layer_b.layer(
            layer_a.layer(inner)
        )
    );
    
    // Request flow: C -> B -> A -> Inner
    // Response flow: Inner -> A -> B -> C
}

Layers are applied in order; the last layer wraps all previous layers.

Type Inference with ServiceBuilder

use tower::ServiceBuilder;
 
// Without ServiceBuilder, types become complex:
fn without_builder() {
    let service = LoggingLayer.layer(
        RateLimitLayer::new(100).layer(
            TimeoutLayer::new(Duration::from_secs(30)).layer(
                MyService::new()
            )
        )
    );
    
    // Type: LoggingService<RateLimitService<TimeoutService<MyService>>>
    // Hard to read, harder to maintain
}
 
// With ServiceBuilder, type is inferred:
fn with_builder() {
    let service = ServiceBuilder::new()
        .timeout(Duration::from_secs(30))
        .rate_limit(100, Duration::from_secs(1))
        .layer(LoggingLayer)
        .service(MyService::new());
    
    // Type is still complex, but you don't have to write it
    // ServiceBuilder handles the composition
}

ServiceBuilder eliminates the need to write complex nested types.

Layers vs ServiceBuilder Methods

use tower::ServiceBuilder;
use tower::Layer;
 
fn built_in_vs_custom() {
    // ServiceBuilder has built-in methods for common layers:
    let service = ServiceBuilder::new()
        .timeout(Duration::from_secs(30))           // Built-in
        .rate_limit(100, Duration::from_secs(1))    // Built-in
        .concurrency_limit(50)                       // Built-in
        .load_shed()                                 // Built-in
        .service(MyService::new());
    
    // Custom layers use .layer():
    let service = ServiceBuilder::new()
        .timeout(Duration::from_secs(30))
        .layer(MyCustomLayer)                        // Custom
        .service(MyService::new());
    
    // .layer() is the escape hatch for any Layer implementation
}

Built-in methods provide convenience; .layer() accepts any Layer.

ServiceBuilder Stack Type

use tower::ServiceBuilder;
 
// ServiceBuilder accumulates layers in its type
fn builder_types() {
    // Each .layer() or built-in method adds to the type stack
    
    let builder1: ServiceBuilder<()> = ServiceBuilder::new();
    
    // After adding timeout:
    let builder2 = builder1.timeout(Duration::from_secs(30));
    // Type: ServiceBuilder<(TimeoutLayer, ())>
    
    // After adding rate limit:
    let builder3 = builder2.rate_limit(100, Duration::from_secs(1));
    // Type: ServiceBuilder<(RateLimitLayer, (TimeoutLayer, ()))>
    
    // The service method finalizes and produces the composed service
}

ServiceBuilder tracks layers in its type for compile-time correctness.

Conditional Layer Application

use tower::ServiceBuilder;
 
fn conditional_layers(enable_logging: bool, enable_timeout: bool) {
    // Manual approach gives explicit control:
    let service = MyService::new();
    
    let service = if enable_timeout {
        TimeoutLayer::new(Duration::from_secs(30)).layer(service)
    } else {
        // Need identity layer or service coercion
        // This is where it gets complex
        service
    };
    
    // ServiceBuilder with condition:
    let mut builder = ServiceBuilder::new();
    
    if enable_timeout {
        builder = builder.timeout(Duration::from_secs(30));
    }
    
    if enable_logging {
        builder = builder.layer(LoggingLayer);
    }
    
    let service = builder.service(MyService::new());
}

Both approaches support conditional layers; manual gives more explicit control.

Layer Identity and Stack Manipulation

use tower::{Layer, ServiceBuilder, Service};
 
// The identity layer does nothing
fn identity_layer() {
    let service = ServiceBuilder::new()
        .layer(tower::layer::util::Identity::new())
        .service(MyService::new());
    
    // Identity is useful for conditional composition:
    // You can replace a layer with Identity to "disable" it
}
 
// Stack manipulation is easier with ServiceBuilder
fn stack_manipulation() {
    // Define layers separately for reuse
    let timeout = TimeoutLayer::new(Duration::from_secs(30));
    let rate_limit = RateLimitLayer::new(100);
    let logging = LoggingLayer;
    
    // Compose with ServiceBuilder
    let builder = ServiceBuilder::new()
        .layer(timeout)
        .layer(rate_limit)
        .layer(logging);
    
    // Apply to any service
    let service1 = builder.service(ServiceA::new());
    let service2 = builder.service(ServiceB::new());
    
    // Or modify the stack:
    let builder2 = ServiceBuilder::new()
        .layer(timeout)
        .layer(rate_limit)
        // Conditionally add logging
        .layer(logging);
}

Layers can be defined separately and combined flexibly.

Implementing Custom Layers

use tower::Layer;
 
// Custom layers implement the Layer trait
pub struct MetricsLayer {
    prefix: String,
}
 
impl MetricsLayer {
    pub fn new(prefix: impl Into<String>) -> Self {
        MetricsLayer {
            prefix: prefix.into(),
        }
    }
}
 
impl<S> Layer<S> for MetricsLayer {
    type Service = MetricsService<S>;
    
    fn layer(&self, inner: S) -> Self::Service {
        MetricsService {
            inner,
            prefix: self.prefix.clone(),
        }
    }
}
 
// Use with ServiceBuilder:
fn use_custom_layer() {
    let service = ServiceBuilder::new()
        .layer(MetricsLayer::new("api"))
        .service(MyService::new());
}

Custom layers integrate seamlessly with ServiceBuilder.

Layer Trait Bounds and Generics

use tower::{Layer, Service};
 
// Layers often need bounds for their services
pub struct RetryLayer {
    max_retries: usize,
}
 
impl<S> Layer<S> for RetryLayer {
    type Service = RetryService<S>;
    
    fn layer(&self, inner: S) -> Self::Service {
        RetryService {
            inner,
            retries: 0,
            max_retries: self.max_retries,
        }
    }
}
 
// The resulting service might need additional bounds:
pub struct RetryService<S> {
    inner: S,
    retries: usize,
    max_retries: usize,
}
 
impl<S, Request> Service<Request> for RetryService<S>
where
    S: Service<Request>,
    S::Error: std::fmt::Debug,  // Additional bounds
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = S::Future;
    
    fn poll_ready(&mut self, cx: &mut std::task::Context<'_>) 
        -> std::task::Poll<Result<(), Self::Error>> 
    {
        self.inner.poll_ready(cx)
    }
    
    fn call(&mut self, request: Request) -> Self::Future {
        // Retry logic would go here
        self.inner.call(request)
    }
}

Layers propagate and potentially add trait bounds.

Real-World Example: Web Service Middleware Stack

use tower::ServiceBuilder;
use std::time::Duration;
 
// Typical web service middleware stack
fn web_service_stack() {
    let service = ServiceBuilder::new()
        // Outermost: handle panics gracefully
        .layer(CatchPanicLayer::new())
        
        // Request tracing
        .layer(TraceLayer::new_for_http())
        
        // Timeouts
        .timeout(Duration::from_secs(30))
        
        // Rate limiting
        .rate_limit(1000, Duration::from_secs(1))
        
        // Concurrency limit
        .concurrency_limit(100)
        
        // Load shedding when overloaded
        .load_shed()
        
        // Innermost: actual service
        .service(ApiHandler::new());
    
    // Request flow:
    // 1. CatchPanic wraps everything
    // 2. Trace logs requests
    // 3. Timeout cancels slow requests
    // 4. RateLimit throttles
    // 5. ConcurrencyLimit bounds concurrent requests
    // 6. LoadShed rejects when at capacity
    // 7. ApiHandler processes the request
}

ServiceBuilder clearly expresses the middleware stack order.

Real-World Example: Reusable Layer Configurations

use tower::ServiceBuilder;
use std::time::Duration;
 
// Define reusable middleware configurations
fn production_layers() -> ServiceBuilder<tower::layer::layer_fn::LayerFn<ProductionStack>> {
    ServiceBuilder::new()
        .timeout(Duration::from_secs(30))
        .rate_limit(1000, Duration::from_secs(1))
        .concurrency_limit(100)
}
 
fn development_layers() -> ServiceBuilder<tower::layer::layer_fn::LayerFn<DevelopmentStack>> {
    ServiceBuilder::new()
        .timeout(Duration::from_secs(60))
        // No rate limiting in development
        .layer(LoggingLayer)  // More verbose logging
}
 
// Use different stacks for different environments
fn configure_service(env: &str) {
    let service = match env {
        "production" => {
            production_layers()
                .layer(MetricsLayer::new("prod"))
                .service(ApiHandler::new())
        }
        "development" => {
            development_layers()
                .service(ApiHandler::new())
        }
        _ => panic!("Unknown environment"),
    };
}

ServiceBuilder enables reusable, composable middleware configurations.

Trade-offs Summary

fn trade_offs() {
    // Layer directly:
    // Pros:
    // - Explicit control over composition
    // - Clear what's happening at each step
    // - Easier to conditionally apply layers
    // - No intermediate builder types
    // - Works with any Layer implementation
    
    // Cons:
    // - Verbose nested calls
    // - Complex type signatures
    // - Harder to reorder layers
    // - Manual type annotations needed
    
    // ServiceBuilder:
    // Pros:
    // - Fluent, readable API
    // - Handles complex nested types
    // - Easy to reorder layers
    // - Built-in methods for common layers
    // - Clean separation of layer config and service
    
    // Cons:
    // - Another abstraction to learn
    // - Builder type accumulates layers
    // - Less explicit about type transformations
    // - May need to extract layers for conditional logic
}

Performance Considerations

use tower::{Layer, ServiceBuilder};
 
fn performance() {
    // Both approaches produce equivalent runtime code
    // ServiceBuilder's "overhead" is only at compile time
    
    // Manual:
    let service1 = LoggingLayer.layer(
        RateLimitLayer::new(100).layer(
            TimeoutLayer::new(Duration::from_secs(30)).layer(
                MyService::new()
            )
        )
    );
    
    // ServiceBuilder:
    let service2 = ServiceBuilder::new()
        .timeout(Duration::from_secs(30))
        .rate_limit(100, Duration::from_secs(1))
        .layer(LoggingLayer)
        .service(MyService::new());
    
    // Both produce identical service implementations
    // ServiceBuilder just makes composition easier at compile time
}

Both approaches produce equivalent runtime code; ServiceBuilder is compile-time sugar.

When to Use Each

use tower::{Layer, ServiceBuilder};
 
fn choosing() {
    // Use Layer directly when:
    // - Building reusable layer implementations
    // - Need precise control over composition order
    // - Conditional layer application is complex
    // - Teaching how middleware works
    // - Working with generic layer implementations
    
    // Use ServiceBuilder when:
    // - Composing many layers
    // - Want readable, maintainable code
    // - Configuring middleware stacks
    // - Creating reusable configurations
    // - Using built-in tower middleware
}

Choose based on complexity, readability needs, and reuse patterns.

Summary Table

fn summary_table() {
    // ┌─────────────────────────────────────────────────────────────────────────┐
    // │ Aspect           │ Layer Direct               │ ServiceBuilder         │
    // ├─────────────────────────────────────────────────────────────────────────┤
    // │ Abstraction      │ Core trait                 │ Fluent wrapper         │
    // │ Composition      │ Manual nesting             │ Chained calls          │
    // │ Types            │ Explicit nested types      │ Inferred internally    │
    // │ Readability      │ Can be verbose             │ Clean and fluent       │
    // │ Flexibility      │ Full control               │ Convenience methods    │
    // │ Custom layers    │ Direct use                 │ Via .layer()           │
    // │ Conditional      │ Natural if/else            │ Builder mutation       │
    // │ Reuse            │ Function composition       │ Builder values         │
    // │ Learning curve   │ Core concept               │ Additional API         │
    // └─────────────────────────────────────────────────────────────────────────┘
}

Key Points Summary

fn key_points() {
    // 1. Layer is the core trait for wrapping services
    // 2. ServiceBuilder provides fluent composition on top of Layer
    // 3. ServiceBuilder handles nested type complexity
    // 4. Layer order matters: last listed is outermost
    // 5. Built-in methods cover common middleware needs
    // 6. .layer() accepts any Layer implementation
    // 7. Both approaches produce equivalent runtime code
    // 8. Use Layer directly for explicit control
    // 9. Use ServiceBuilder for readable stacks
    // 10. ServiceBuilder enables reusable configurations
    // 11. Conditional layers work with both approaches
    // 12. Custom layers integrate seamlessly with ServiceBuilder
}

Key insight: Layer is the fundamental building block for middleware, while ServiceBuilder is ergonomics on top. Every ServiceBuilder method ultimately calls .layer() internally. The choice is between explicit, transparent composition (Layer directly) and readable, maintainable configuration (ServiceBuilder). For complex middleware stacks with many layers, ServiceBuilder's fluent API significantly improves code clarity. For simple cases or when teaching middleware concepts, direct Layer usage shows exactly what's happening. The two approaches compose perfectly: you can use ServiceBuilder for most composition and drop down to direct Layer calls when you need precise control.