Arc<Mutex>

Guides

Rust walkthroughs

← Back to all guides

What is the purpose of criterion::PlottingBackend for customizing benchmark visualization output?

PlottingBackend configures how Criterion generates benchmark charts and graphs, allowing you to choose between different visualization backends like gnuplot or plotters, disable plotting entirely, or customize output formats—controlling whether benchmarks produce visual output and in what form. The key purpose is flexibility: different environments have different visualization capabilities, and PlottingBackend lets you adapt Criterion's output to match your system's available tools or CI pipeline requirements.

Basic PlottingBackend Usage

use criterion::{Criterion, PlottingBackend};
 
fn basic_plotting_backend() {
    // Default: Criterion auto-detects available backend
    let mut criterion = Criterion::default();
    
    // This uses the default plotting backend
    // Criterion will try gnuplot first, then fall back to plotters
    criterion.bench_function("example", |b| {
        b.iter(|| {
            // Benchmark code here
            (0..1000).sum::<u64>()
        })
    });
}
 
fn explicit_backend() {
    // Explicitly specify the plotting backend
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Gnuplot);
    
    criterion.bench_function("example", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // Now uses gnuplot for all plots
}

PlottingBackend configures which visualization library Criterion uses for benchmark charts.

Available Plotting Backends

use criterion::{Criterion, PlottingBackend};
 
fn available_backends() {
    // Gnuplot backend
    // - Traditional backend
    // - Requires gnuplot installed
    // - Produces SVG and PNG outputs
    // - Most feature-complete
    let gnuplot = Criterion::default()
        .plotting_backend(PlottingBackend::Gnuplot);
    
    // Plotters backend
    // - Pure Rust implementation
    // - No external dependencies
    // - Works in environments without gnuplot
    // - Good for CI/CD environments
    let plotters = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters);
    
    // Auto backend (default)
    // - Tries gnuplot first
    // - Falls back to plotters if gnuplot unavailable
    let auto_backend = Criterion::default()
        .plotting_backend(PlottingBackend::Auto);
    
    // Note: PlottingBackend is configured per-Criterion instance
    // All benchmarks in this group use the specified backend
}

Criterion supports gnuplot (external tool) and plotters (pure Rust), with auto-detection available.

Disabling Plotting

use criterion::{Criterion, PlottingBackend};
 
fn disable_plotting() {
    // In CI environments or when you don't need visual output
    // disable plotting to avoid dependency issues
    
    let mut criterion = Criterion::default()
        .without_plots();
    
    // This disables all plotting
    // Benchmark results still printed to console
    // No charts generated
    
    criterion.bench_function("no_plots", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // Alternative: Use PlottingBackend::None if available
    // The without_plots() method is the idiomatic way
}

Use without_plots() when you only need console output, such as in CI environments.

Gnuplot Backend Details

use criterion::{Criterion, PlottingBackend};
 
fn gnuplot_backend() {
    // Gnuplot backend characteristics:
    // - External dependency: requires gnuplot installed
    // - Produces high-quality SVG and PNG charts
    // - Interactive HTML reports possible
    // - Best chart quality and features
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Gnuplot);
    
    // Configure output directory
    criterion = criterion.output_directory("target/criterion");
    
    // Gnuplot generates:
    // - SVG files for each benchmark
    // - PNG files as alternative
    // - HTML report with all benchmarks
    
    criterion.bench_function("example", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // Check for gnuplot availability:
    // Criterion will error if gnuplot is not installed
    // when using PlottingBackend::Gnuplot explicitly
}

Gnuplot provides the best chart quality but requires external installation.

Plotters Backend Details

use criterion::{Criterion, PlottingBackend};
 
fn plotters_backend() {
    // Plotters backend characteristics:
    // - Pure Rust implementation
    // - No external dependencies
    // - Works everywhere Rust compiles
    // - Good quality charts
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters);
    
    criterion.bench_function("example", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // Plotters is ideal for:
    // - CI/CD pipelines without gnuplot
    // - Docker containers
    // - Cross-platform consistency
    // - Environments where installing gnuplot is difficult
    
    // Note: Some advanced chart features may not be available
    // compared to gnuplot backend
}

Plotters is the pure Rust option, working anywhere without external dependencies.

Auto-Detection Behavior

use criterion::{Criterion, PlottingBackend};
 
fn auto_detection() {
    // PlottingBackend::Auto (default behavior):
    // 1. Check if gnuplot is available
    // 2. If yes, use Gnuplot backend
    // 3. If no, fall back to Plotters backend
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Auto);
    
    // This is the default, so you can omit it:
    let mut criterion = Criterion::default();  // Uses Auto
    
    // Auto-detection ensures:
    // - Local development: Uses gnuplot if available (best quality)
    // - CI/CD: Falls back to plotters (no external dependency)
    // - Works in all environments
    
    // To check which backend was actually used:
    // Criterion logs this information during benchmark execution
}

Auto mode ensures benchmarks work in any environment by detecting available tools.

CI/CD Integration

use criterion::{Criterion, PlottingBackend};
 
fn ci_integration() {
    // In CI environments, plotting may not be needed
    // Or you may want plotters for consistency
    
    // Option 1: Disable plots entirely
    let mut criterion = Criterion::default()
        .without_plots();
    
    // Only console output, no charts
    // Fastest option, no visualization dependencies
    
    // Option 2: Use plotters (works without gnuplot)
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters);
    
    // Charts generated, no external dependency
    // Works in Docker, GitHub Actions, etc.
    
    // Option 3: Install gnuplot in CI for best quality
    // GitHub Actions example:
    // - run: sudo apt-get install gnuplot
    // Then use PlottingBackend::Gnuplot
    
    criterion.bench_function("ci_bench", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
}

For CI, use without_plots() or PlottingBackend::Plotters to avoid gnuplot dependency issues.

Custom Output Directory

use criterion::{Criterion, PlottingBackend};
 
fn custom_output_directory() {
    // PlottingBackend works with output directory configuration
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters)
        .output_directory("benchmark_results");
    
    // Charts are written to benchmark_results/
    // Default is target/criterion/
    
    criterion.bench_function("example", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // Output files:
    // benchmark_results/
    //   example/
    //     new/
    //       report/index.html  (if using plotters)
    //       report/svg/
    //       ...
    
    // The plotting backend determines the file formats
    // Gnuplot: SVG + PNG + HTML
    // Plotters: SVG + HTML
}

Configure output directory alongside plotting backend for custom benchmark artifact locations.

Comparing Backend Output

use criterion::{Criterion, PlottingBackend};
 
fn compare_backend_output() {
    // Both backends produce similar chart types:
    // - Sample time distribution (histogram)
    // - Regression analysis
    // - Comparison with previous runs
    // - Throughput charts
    
    // Gnuplot advantages:
    // - More polished appearance
    // - Additional chart types
    // - Interactive HTML reports
    // - Customizable themes
    
    // Plotters advantages:
    // - Pure Rust (no external dependency)
    // - Consistent cross-platform output
    // - Works in constrained environments
    
    // Example comparison:
    
    // With gnuplot (requires: apt-get install gnuplot)
    let mut c_gnuplot = Criterion::default()
        .plotting_backend(PlottingBackend::Gnuplot);
    
    c_gnuplot.bench_function("with_gnuplot", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // With plotters (no external dependency)
    let mut c_plotters = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters);
    
    c_plotters.bench_function("with_plotters", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
}

Both backends produce similar visualizations; gnuplot has more features, plotters has wider compatibility.

Environment-Specific Configuration

use criterion::{Criterion, PlottingBackend};
use std::env;
 
fn environment_specific() {
    // Adapt plotting based on environment
    let plotting_backend = if env::var("CI").is_ok() {
        // In CI: Use plotters or disable plots
        match env::var("CI_PLOTS").as_deref() {
            Ok("true") => PlottingBackend::Plotters,
            _ => {
                // Disable plots in CI by default
                let mut criterion = Criterion::default().without_plots();
                criterion.bench_function("ci_test", |b| {
                    b.iter(|| (0..1000).sum::<u64>())
                });
                return;
            }
        }
    } else {
        // Local development: Use gnuplot if available
        PlottingBackend::Auto
    };
    
    let mut criterion = Criterion::default()
        .plotting_backend(plotting_backend);
    
    criterion.bench_function("adaptive", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
}

Adapt backend selection to environment: gnuplot for local development, plotters or no plots for CI.

Benchmark Group Configuration

use criterion::{Criterion, PlottingBackend, BenchmarkId, BatchSize};
 
fn group_configuration() {
    // PlottingBackend is set at Criterion level
    // All groups inherit the same backend
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters);
    
    // First group
    let mut group1 = criterion.benchmark_group("sorting");
    
    for size in [100, 1000, 10000] {
        group1.bench_with_input(
            BenchmarkId::new("sort", size),
            size,
            |b, &size| {
                let mut data: Vec<u64> = (0..size).rev().collect();
                b.iter_batched(
                    || data.clone(),
                    |mut d| d.sort(),
                    BatchSize::SmallInput,
                );
            },
        );
    }
    
    group1.finish();
    
    // Second group - same backend
    let mut group2 = criterion.benchmark_group("parsing");
    
    group2.bench_function("json", |b| {
        b.iter(|| {
            let data = r#"{"key": "value"}"#;
            serde_json::from_str::<serde_json::Value>(data).unwrap()
        });
    });
    
    group2.finish();
    
    // All charts use Plotters backend
}

Plotting backend is configured per-Criterion instance, applying to all benchmark groups.

Generated Output Files

use criterion::{Criterion, PlottingBackend};
 
fn output_files() {
    // Understanding what files are generated
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Plotters)
        .output_directory("target/criterion");
    
    criterion.bench_function("my_bench", |b| {
        b.iter(|| (0..1000).sum::<u64>())
    });
    
    // Generated files structure:
    // target/criterion/
    //   my_bench/
    //     new/
    //       sample.json         (raw sample data)
    //       estimates.json      (statistical estimates)
    //       benchmark.json      (benchmark metadata)
    //       report/
    //         index.html        (main report page)
    //         svg/              (SVG charts)
    //           sample.svg
    //           ...
    
    // With Gnuplot backend:
    //   - Additional PNG files
    //   - More detailed SVG charts
    //   - Interactive HTML features
    
    // With Plotters backend:
    //   - SVG files only
    //   - Simpler HTML report
    //   - No PNG output
}

Both backends generate similar file structures with charts and statistical data.

Checking Backend Availability

use criterion::{Criterion, PlottingBackend};
 
fn check_availability() {
    // Criterion handles backend availability gracefully
    
    // With PlottingBackend::Auto:
    // - Tries gnuplot first
    // - Falls back to plotters
    // - No errors if gnuplot missing
    
    let mut criterion = Criterion::default()
        .plotting_backend(PlottingBackend::Auto);
    
    // With explicit PlottingBackend::Gnuplot:
    // - Errors if gnuplot not installed
    // - Clear error message
    
    // Use Auto for portable benchmarks
    // Use explicit Gnuplot/Plotters when you need specific backend
    
    // To check if gnuplot is available before running:
    // This is typically done by Criterion internally
    // You can also check manually:
    let gnuplot_available = which::which("gnuplot").is_ok();
    
    let backend = if gnuplot_available {
        PlottingBackend::Gnuplot
    } else {
        PlottingBackend::Plotters
    };
    
    let mut criterion = Criterion::default()
        .plotting_backend(backend);
}

Use PlottingBackend::Auto for portable code that adapts to available tools.

Synthesis

Core purpose:

// PlottingBackend controls how Criterion generates visual output
let criterion = Criterion::default()
    .plotting_backend(PlottingBackend::Plotters);
 
// Options:
// - Gnuplot: Best quality, requires external tool
// - Plotters: Pure Rust, works everywhere
// - Auto: Detect best available
// - without_plots(): Disable charts entirely

Backend comparison:

Feature Gnuplot Plotters None
External dependency Yes No No
Chart quality Excellent Good N/A
CI compatibility Requires setup Works Works
Output formats SVG, PNG, HTML SVG, HTML Console only

Common patterns:

// Local development (best quality)
Criterion::default()
    .plotting_backend(PlottingBackend::Auto)  // Default
 
// CI environment (no gnuplot)
Criterion::default()
    .plotting_backend(PlottingBackend::Plotters)
 
// CI environment (console only)
Criterion::default()
    .without_plots()
 
// Explicit gnuplot (error if missing)
Criterion::default()
    .plotting_backend(PlottingBackend::Gnuplot)

Key insight: PlottingBackend exists to make Criterion's visual output portable across environments—local development machines with gnuplot installed get the full-featured gnuplot charts, while CI pipelines and constrained environments can use the pure-Rust plotters backend or disable plotting entirely. The Auto backend provides graceful fallback: if gnuplot is available, use it for better charts; if not, use plotters which has no external dependencies. For CI pipelines that only care about benchmark numbers and not visual output, without_plots() eliminates all plotting overhead and dependency concerns. The choice of backend doesn't affect benchmark accuracy—only the presentation of results—so you can run the same benchmarks with different backends in different environments without changing benchmark code. Use Gnuplot when you control the environment and want the best output; use Plotters for portability; use Auto for adaptable code; use without_plots() for pure numerical benchmarking.