How does `bytes::BufMut` enable efficient buffer writing without reallocation?

bytes::BufMut is a trait for writing bytes into a buffer with automatic growth management, allowing efficient appends without manual reallocation tracking. Unlike Vec<u8> which grows by powers of two and may reallocate unpredictably, BufMut implementations like BytesMut provide more control over allocation strategy and avoid copies when possible. The key advantage is the remaining_mut() method that reports available capacity, and the advance_mut() method that commits written bytes without moving data. This enables patterns like reserving capacity once, writing multiple times, and growing only when necessary—all while tracking the valid written portion separately from capacity.

Basic BufMut Usage

use bytes::{BufMut, BytesMut};
 
fn main() {
    let mut buf = BytesMut::new();
    
    // Write bytes - buffer grows automatically
    buf.put_slice(b"hello");
    buf.put_slice(b" ");
    buf.put_slice(b"world");
    
    // Convert to Bytes (zero-copy possible)
    let bytes = buf.freeze();
    println!("{}", String::from_utf8_lossy(&bytes));
}

BufMut::put_slice writes bytes, growing the buffer if necessary.

Vec vs BufMut

use bytes::{BufMut, BytesMut};
 
fn main() {
    // Vec<u8> approach - manual growth
    let mut vec = Vec::new();
    vec.write_all(b"hello").unwrap(); // Would need std::io::Write
    vec.extend_from_slice(b"world");
    
    // BufMut approach - automatic growth with tracking
    let mut buf = BytesMut::new();
    buf.put_slice(b"hello");
    buf.put_slice(b"world");
    
    // Key difference: BufMut tracks remaining capacity
    println!("Vec capacity: {}", vec.capacity());
    println!("BufMut remaining: {}", buf.remaining_mut());
}

BufMut tracks remaining capacity explicitly.

Checking Remaining Capacity

use bytes::{BufMut, BytesMut};
 
fn main() {
    let mut buf = BytesMut::with_capacity(64);
    
    println!("Initial remaining: {}", buf.remaining_mut());
    // remaining_mut() returns how many more bytes can be written
    
    buf.put_slice(b"hello");
    println!("After write: {}", buf.remaining_mut());
    // Decreased by 5 bytes
    
    // Check before writing large amounts
    if buf.remaining_mut() >= 1024 {
        buf.put_slice(&[0u8; 1024]);
    } else {
        // Reserve more space first
        buf.reserve(1024);
        buf.put_slice(&[0u8; 1024]);
    }
}

remaining_mut() lets you check capacity before writing.

Reserving Capacity

use bytes::{BufMut, BytesMut};
 
fn main() {
    let mut buf = BytesMut::new();
    
    // Reserve capacity for known size
    buf.reserve(1024);
    
    // Now we can write without triggering growth
    println!("Reserved remaining: {}", buf.remaining_mut());
    
    buf.put_slice(&[0u8; 512]);
    println!("After partial write: {}", buf.remaining_mut());
    
    // reserve() grows the buffer but doesn't clear existing data
    buf.reserve(2048);
    println!("After more reserve: {}", buf.remaining_mut());
}

reserve() pre-allocates space to avoid incremental growth.

Direct Write with advance_mut

use bytes::{BufMut, BytesMut};
use std::io::{self, Write};
 
fn main() -> io::Result<()> {
    let mut buf = BytesMut::with_capacity(64);
    
    // Get direct access to uninitialized memory
    // This is unsafe because you must initialize the bytes
    unsafe {
        let remaining = buf.remaining_mut();
        let dst = buf.bytes_mut();
        
        // Write directly into buffer
        let written = {
            let mut cursor = io::Cursor::new(dst);
            write!(cursor, "Hello, world!")?;
            cursor.position() as usize
        };
        
        // Commit the written bytes
        buf.advance_mut(written);
    }
    
    println!("Written: {}", String::from_utf8_lossy(&buf));
    Ok(())
}

advance_mut commits bytes written directly into the buffer.

Writing Different Types

use bytes::{BufMut, BytesMut, BigEndian, LittleEndian};
 
fn main() {
    let mut buf = BytesMut::new();
    
    // Integers in big-endian
    buf.put_u16(0x1234);
    buf.put_u32(0x12345678);
    buf.put_u64(0x1234567890ABCDEF);
    
    // Integers in little-endian
    buf.put_u16_le(0x1234);
    buf.put_u32_le(0x12345678);
    buf.put_u64_le(0x1234567890ABCDEF);
    
    // Floats
    buf.put_f32(3.14159);
    buf.put_f64(2.71828);
    
    // Bytes
    buf.put_u8(0xFF);
    buf.put_i8(-1);
    
    println!("Buffer length: {}", buf.len());
}

BufMut provides type-safe methods for writing integers and floats.

Extending from BufMut

use bytes::{BufMut, BytesMut, Buf};
 
fn main() {
    let mut buf = BytesMut::new();
    
    // put_slice for byte slices
    buf.put_slice(b"hello");
    
    // put for other Buf types
    let other = BytesMut::from("world");
    buf.put(other);
    
    // put_bytes for repeating
    buf.put_bytes(b'-', 10);
    
    println!("{}", String::from_utf8_lossy(&buf));
}

put works with any type implementing Buf.

Growing Strategy

use bytes::{BufMut, BytesMut};
 
fn main() {
    let mut buf = BytesMut::new();
    
    // First growth
    buf.put_slice(&[0u8; 100]);
    println!("After 100: capacity={}, remaining={}", 
        buf.capacity(), buf.remaining_mut());
    
    // Second growth
    buf.put_slice(&[0u8; 100]);
    println!("After 200: capacity={}, remaining={}", 
        buf.capacity(), buf.remaining_mut());
    
    // BytesMut grows as needed, typically doubling
    // But it can also reuse allocated space after split
    
    // Split off consumed portion
    let _consumed = buf.split();
    
    // Now buf is empty but may have reserved capacity
    println!("After split: capacity={}, remaining={}", 
        buf.capacity(), buf.remaining_mut());
}

BytesMut grows automatically but can reuse capacity.

Split and Freeze

use bytes::{BufMut, BytesMut};
 
fn main() {
    let mut buf = BytesMut::new();
    buf.put_slice(b"hello");
    buf.put_slice(b"world");
    
    // split() returns the filled portion and clears buf
    let filled = buf.split();
    // buf is now empty, but capacity may be retained
    
    buf.put_slice(b"next");
    // May reuse capacity from before
    
    // freeze() converts to immutable Bytes (zero-copy)
    let bytes = buf.freeze();
    // buf is now consumed, bytes holds the data
    
    // split_to() for partial split
    let mut buf2 = BytesMut::new();
    buf2.put_slice(b"hello world");
    let prefix = buf2.split_to(5);
    
    println!("Prefix: {:?}", String::from_utf8_lossy(&prefix));
    println!("Remaining: {:?}", String::from_utf8_lossy(&buf2));
}

split() and freeze() enable zero-copy extraction.

Resizing vs Growing

use bytes::{BufMut, BytesMut};
 
fn main() {
    let mut buf = BytesMut::with_capacity(100);
    
    // Write some data
    buf.put_slice(b"hello");
    
    // resize() changes length (pads with zeros)
    buf.resize(20, 0);
    println!("After resize: {:?}", &buf[..]);
    
    // truncate() reduces length
    buf.truncate(5);
    println!("After truncate: {:?}", &buf[..]);
    
    // clear() resets length to 0, keeps capacity
    buf.clear();
    println!("After clear: len={}, cap={}", buf.len(), buf.capacity());
}

BufMut supports resizing operations while maintaining capacity.

Custom BufMut Implementation

use bytes::{BufMut, BytesMut};
 
// A wrapper that tracks write statistics
struct TrackingBufMut {
    inner: BytesMut,
    bytes_written: usize,
    growths: usize,
}
 
impl TrackingBufMut {
    fn new() -> Self {
        Self {
            inner: BytesMut::new(),
            bytes_written: 0,
            growths: 0,
        }
    }
    
    fn write(&mut self, data: &[u8]) {
        let remaining = self.inner.remaining_mut();
        if remaining < data.len() {
            self.growths += 1;
        }
        self.inner.put_slice(data);
        self.bytes_written += data.len();
    }
    
    fn stats(&self) -> (usize, usize) {
        (self.bytes_written, self.growths)
    }
}
 
fn main() {
    let mut buf = TrackingBufMut::new();
    
    buf.write(b"hello");
    buf.write(&[0u8; 100]);
    
    let (written, growths) = buf.stats();
    println!("Written: {}, Growth events: {}", written, growths);
}

You can implement BufMut for custom types.

Buffer Reuse

use bytes::{BufMut, BytesMut};
 
fn process_messages() {
    let mut buf = BytesMut::with_capacity(1024);
    
    // Process multiple messages reusing buffer
    for i in 0..10 {
        buf.clear(); // Reset length, keep capacity
        buf.put_slice(format!("Message {}", i).as_bytes());
        buf.put_u8(b'\n');
        
        // Process the message
        let message = buf.split();
        println!("Processing: {:?}", String::from_utf8_lossy(&message));
    }
    
    // Only one allocation for 10 messages
}

Reusing BytesMut avoids repeated allocations.

Comparison with Vec

use bytes::{BufMut, BytesMut};
 
fn main() {
    // Vec<u8> - manual management
    let mut vec = Vec::new();
    vec.reserve(100);
    vec.extend_from_slice(b"hello");
    vec.extend_from_slice(b"world");
    // Converting to shared reference might copy
    
    // BytesMut - automatic tracking
    let mut buf = BytesMut::new();
    buf.put_slice(b"hello");
    buf.put_slice(b"world");
    // freeze() gives Bytes without copy
    let bytes = buf.freeze();
    
    // Key differences:
    // 1. BytesMut tracks written vs capacity
    // 2. freeze() enables zero-copy sharing
    // 3. split() returns consumed portion efficiently
    // 4. remaining_mut() queries available space
}

BytesMut is designed for network buffers with efficient sharing.

Working with IoSlice

use bytes::{BufMut, BytesMut};
use std::io::IoSlice;
 
fn main() {
    let mut buf1 = BytesMut::with_capacity(64);
    let mut buf2 = BytesMut::with_capacity(64);
    
    buf1.put_slice(b"First buffer");
    buf2.put_slice(b"Second buffer");
    
    // Convert to IoSlice for vectored I/O
    let slices: Vec<IoSlice> = vec![
        IoSlice::new(&buf1),
        IoSlice::new(&buf2),
    ];
    
    // Can use with write_vectored
    println!("First: {:?}", String::from_utf8_lossy(&buf1));
    println!("Second: {:?}", String::from_utf8_lossy(&buf2));
}

BytesMut integrates with vectored I/O operations.

Chunk-Based Writing

use bytes::{BufMut, BytesMut, Buf};
 
fn main() {
    let mut buf = BytesMut::new();
    
    // Write in chunks
    let data = b"hello world, this is a longer message";
    let chunk_size = 8;
    
    for chunk in data.chunks(chunk_size) {
        if buf.remaining_mut() < chunk.len() {
            buf.reserve(chunk.len());
        }
        buf.put_slice(chunk);
    }
    
    println!("Written: {:?}", String::from_utf8_lossy(&buf));
}

Write in chunks while managing capacity efficiently.

Real-World Example: Protocol Buffer

use bytes::{BufMut, BytesMut, BigEndian};
 
// A simple message: [length: u32][type: u8][payload: bytes]
fn encode_message(msg_type: u8, payload: &[u8]) -> BytesMut {
    let total_len = 1 + payload.len(); // type + payload
    let mut buf = BytesMut::with_capacity(4 + total_len);
    
    // Length header
    buf.put_u32::<BigEndian>(total_len as u32);
    
    // Message type
    buf.put_u8(msg_type);
    
    // Payload
    buf.put_slice(payload);
    
    buf
}
 
fn main() {
    let msg = encode_message(0x01, b"Hello, world!");
    
    println!("Encoded {} bytes", msg.len());
    
    // Can be sent directly over network
    // Or shared via freeze() without copying
}

BufMut is ideal for encoding network protocols.

Real-World Example: HTTP Response

use bytes::{BufMut, BytesMut};
 
fn build_http_response(status: u16, headers: &[(String, String)], body: &[u8]) -> BytesMut {
    let mut buf = BytesMut::new();
    
    // Status line
    buf.put_slice(format!("HTTP/1.1 {} OK\r\n", status).as_bytes());
    
    // Headers
    for (name, value) in headers {
        buf.put_slice(format!("{}: {}\r\n", name, value).as_bytes());
    }
    
    // Empty line
    buf.put_slice(b"\r\n");
    
    // Body
    buf.put_slice(body);
    
    buf
}
 
fn main() {
    let response = build_http_response(
        200,
        &[
            ("Content-Type".to_string(), "text/plain".to_string()),
            ("Content-Length".to_string(), "5".to_string()),
        ],
        b"Hello",
    );
    
    println!("{}", String::from_utf8_lossy(&response));
}

Building responses with BufMut avoids repeated string allocations.

Capacity Management Strategy

use bytes::{BufMut, BytesMut};
 
struct SmartBuffer {
    buf: BytesMut,
    growth_factor: usize,
    min_capacity: usize,
}
 
impl SmartBuffer {
    fn new() -> Self {
        Self {
            buf: BytesMut::new(),
            growth_factor: 2,
            min_capacity: 64,
        }
    }
    
    fn ensure_capacity(&mut self, additional: usize) {
        let remaining = self.buf.remaining_mut();
        if remaining < additional {
            // Grow by factor, but at least enough for additional
            let current = self.buf.capacity();
            let needed = self.buf.len() + additional;
            let new_capacity = (needed * self.growth_factor)
                .max(self.min_capacity);
            
            self.buf.reserve(new_capacity - current);
        }
    }
    
    fn write(&mut self, data: &[u8]) {
        self.ensure_capacity(data.len());
        self.buf.put_slice(data);
    }
}
 
fn main() {
    let mut smart = SmartBuffer::new();
    smart.write(b"hello");
    smart.write(&[0u8; 100]);
}

Custom capacity strategies prevent allocation surprises.

Thread Safety

use bytes::{BufMut, BytesMut, Bytes};
 
fn main() {
    let mut buf = BytesMut::new();
    buf.put_slice(b"hello");
    
    // freeze() returns Bytes which is Clone + Send + Sync
    let bytes: Bytes = buf.freeze();
    
    // Can be sent to other threads
    let bytes2 = bytes.clone();
    let bytes3 = bytes.clone();
    
    // Each clone references the same underlying data
    println!("{:?}", bytes2);
    println!("{:?}", bytes3);
}

freeze() produces immutable, shareable Bytes.

Zero-Copy Patterns

use bytes::{BufMut, BytesMut, Buf, Bytes};
 
fn main() {
    // Chain buffers without copying
    let mut buf1 = BytesMut::new();
    buf1.put_slice(b"hello ");
    
    let mut buf2 = BytesMut::new();
    buf2.put_slice(b"world");
    
    // Create Bytes references
    let b1: Bytes = buf1.freeze();
    let b2: Bytes = buf2.freeze();
    
    // Chain (no copy)
    let chain = b1.chain(b2);
    println!("Chained: {:?}", chain.bytes());
    
    // Copying only when necessary
    let mut combined = BytesMut::with_capacity(b1.len() + b2.len());
    combined.put_slice(&b1);
    combined.put_slice(&b2);
    println!("Combined: {:?}", String::from_utf8_lossy(&combined));
}

Bytes can be chained without copying.

Synthesis

BufMut core concepts:

remaining_mut() - available capacity
put_* methods - write typed data
reserve() - pre-allocate capacity
advance_mut() - commit direct writes
split() - extract written data
freeze() - create immutable Bytes

Efficiency mechanisms:

Tracks written length separately from capacity
Grows only when remaining_mut() is insufficient
split() returns filled portion without copy
freeze() creates shareable Bytes without copy
Capacity can be reused after clear() or split()

When to use BufMut:

Network protocol encoding
Building messages incrementally
Scenarios needing zero-copy sharing
Reusable buffer pools
Vectored I/O preparation

When Vec is sufficient:

Simple byte collection
No sharing requirements
Standard Write trait usage
Simpler use cases

Key insight: BufMut is about separating the concepts of "written length" and "capacity" while providing a path to immutable shared ownership. Vec<u8> conflates these—a slice of Vec either borrows the whole Vec (preventing mutation) or requires copying. BytesMut solves this by allowing split() to extract the written portion, leaving capacity behind for reuse, and freeze() to create a Bytes that can be cloned cheaply (reference counting, not copying). This design matches network programming patterns: write into a buffer, send the data (shared), and reuse the buffer for the next message. The remaining_mut() method enables checking capacity before writing, avoiding the "write then check" pattern that could leave partial writes.

How does bytes::BufMut enable efficient buffer writing without reallocation?