What is the purpose of `bytes::Buf::chunk` for accessing underlying buffer slices without copying?

chunk() returns a &[u8] slice pointing directly to the next contiguous portion of the buffer's internal storage, without copying data or advancing the buffer's cursor. This enables zero-copy reads when you need to inspect or process buffer contents without consuming them. For simple buffer types like Bytes or BytesMut, chunk() typically returns the entire remaining content. For composite buffers like Chain (two buffers concatenated) or Take (a limited view), chunk() returns only the first contiguous segment, requiring multiple calls with advance() to traverse all data. The key insight is that chunk() provides a view into the underlying storage—you can read the data without allocating or copying, which is critical for high-performance I/O processing where buffer copies would dominate CPU usage.

Basic chunk Usage

use bytes::Bytes;
use bytes::Buf;
 
fn main() {
    let data = Bytes::from("hello world");
    
    // chunk() returns a slice into the buffer's storage
    let chunk = data.chunk();
    
    println!("Chunk: {:?}", chunk);  // b"hello world"
    println!("Length: {}", chunk.len());
    
    // The buffer is not advanced or modified
    assert_eq!(data.chunk(), b"hello world");
    assert_eq!(data.remaining(), 11);  // Still has all data
    
    // This is a zero-copy operation
    // The slice points directly into the Bytes' internal storage
}

chunk() returns a reference to the buffer's internal data without copying or advancing.

chunk with Buffer Cursor and advance

use bytes::Bytes;
use bytes::Buf;
 
fn main() {
    let mut data = Bytes::from("hello world");
    
    // chunk() shows remaining data from cursor position
    assert_eq!(data.chunk(), b"hello world");
    assert_eq!(data.remaining(), 11);
    
    // advance() moves the cursor, chunk() then shows less data
    data.advance(6);  // Skip "hello "
    
    assert_eq!(data.chunk(), b"world");
    assert_eq!(data.remaining(), 5);
    
    // advance() doesn't copy; it just moves internal pointers
    data.advance(2);  // Skip "wo"
    
    assert_eq!(data.chunk(), b"rld");
    assert_eq!(data.remaining(), 3);
    
    // chunk() always returns the remaining contiguous portion
    // from the current cursor position
}

chunk() returns data from the current cursor position; advance() moves the cursor forward.

chunk with BytesMut for Mutation

use bytes::BytesMut;
use bytes::Buf;
 
fn main() {
    let mut data = BytesMut::from("hello world");
    
    // For BytesMut, chunk() returns an immutable slice
    let chunk = data.chunk();
    assert_eq!(chunk, b"hello world");
    
    // To mutate, use chunk_mut() which returns &mut [u8]
    // (only available on BytesMut, not on the Buf trait)
    let chunk_mut = data.chunk_mut();
    chunk_mut[0] = b'H';  // Capitalize first letter
    
    // Now chunk() shows the modified data
    assert_eq!(data.chunk(), b"Hello world");
    
    // advance() works the same way
    data.advance(6);
    assert_eq!(data.chunk(), b"world");
    
    // BytesMut can also grow
    data.extend_from_slice(b"!");
    assert_eq!(data.chunk(), b"world!");
}

BytesMut::chunk() gives immutable access; chunk_mut() provides mutable access for modification.

Composite Buffers with Chain

use bytes::{Bytes, Buf};
use bytes::buf::Chain;
 
fn main() {
    let buf1 = Bytes::from("hello ");
    let buf2 = Bytes::from("world");
    
    // Chain two buffers together
    let mut chained: Chain<Bytes, Bytes> = buf1.chain(buf2);
    
    // chunk() only returns the FIRST buffer's portion
    // This is because Chain is a non-contiguous buffer
    assert_eq!(chained.chunk(), b"hello ");
    assert_eq!(chained.remaining(), 11);  // Total remaining
    
    // To access the second buffer's chunk, advance past the first
    chained.advance(6);
    
    // Now chunk() returns the second buffer's portion
    assert_eq!(chained.chunk(), b"world");
    assert_eq!(chained.remaining(), 5);
    
    // This pattern is useful for processing data from multiple sources
    // without copying them into a single contiguous buffer
}

For composite buffers, chunk() returns only the first contiguous segment; use advance() to traverse.

Iterating Over All Chunks

use bytes::{Bytes, Buf};
use bytes::buf::Chain;
 
fn main() {
    let buf1 = Bytes::from("alpha-");
    let buf2 = Bytes::from("beta-");
    let buf3 = Bytes::from("gamma");
    
    // Chain multiple buffers
    let mut chained = buf1.chain(buf2).chain(buf3);
    
    // Process each chunk without copying
    let mut all_chunks = Vec::new();
    while chained.has_remaining() {
        let chunk = chained.chunk();
        all_chunks.push(std::str::from_utf8(chunk).unwrap().to_string());
        println!("Processing chunk: {:?}", chunk);
        chained.advance(chunk.len());
    }
    
    assert_eq!(all_chunks, vec!["alpha-", "beta-", "gamma"]);
    
    // Alternative: use chunks() iterator
    let buf1 = Bytes::from("one-");
    let buf2 = Bytes::from("two-");
    let buf3 = Bytes::from("three");
    let chained = buf1.chain(buf2).chain(buf3);
    
    let chunks: Vec<&[u8]> = chained.chunk_iter().collect();
    println!("All chunks: {:?}", chunks);
    
    // Note: chunk_iter() is available in newer versions of bytes
}

Iterate over chunks with has_remaining() and advance() to process non-contiguous buffers.

Comparison with copy_to_bytes

use bytes::{Bytes, Buf};
 
fn main() {
    let data = Bytes::from("hello world");
    
    // chunk() - zero-copy, returns a reference
    let chunk: &[u8] = data.chunk();
    // chunk points into data's storage
    
    // copy_to_bytes() - copies data into new Bytes
    let mut data2 = Bytes::from("hello world");
    let copied: Bytes = data2.copy_to_bytes(5);
    // copied is a NEW allocation containing "hello"
    // data2 is advanced and now contains " world"
    
    assert_eq!(copied, Bytes::from("hello"));
    assert_eq!(data2.chunk(), b" world");
    
    // copy_to_bytes() is useful when you need an owned Bytes
    // chunk() is useful when you just need to read
    
    // For large data, chunk() is much more efficient
    let large_data = Bytes::from(vec![0u8; 1_000_000]);
    
    // This just returns a reference - O(1)
    let _chunk = large_data.chunk();
    
    // This would copy 1MB - O(n)
    // let _copied = large_data.copy_to_bytes(1_000_000);
}

chunk() is zero-copy; copy_to_bytes() allocates and copies data into a new Bytes.

Reading Typed Data with chunk

use bytes::{Bytes, Buf, BufMut};
 
fn main() {
    // Create a buffer with some binary data
    let mut data = bytes::BytesMut::new();
    data.put_u32(0x12345678);  // 4 bytes
    data.put_u16(0xABCD);       // 2 bytes
    data.put_u8(0xEF);          // 1 byte
    
    let mut buf = data.freeze();  // Convert to immutable Bytes
    
    // Use chunk() to inspect without consuming
    let chunk = buf.chunk();
    println!("Raw bytes: {:02X?}", chunk);
    // [12, 34, 56, 78, AB, CD, EF]
    
    // Now consume using typed readers
    let value_u32 = buf.get_u32();  // Consumes 4 bytes
    println!("u32: {:08X}", value_u32);
    
    // chunk() now shows remaining data
    assert_eq!(buf.chunk(), &[0xAB, 0xCD, 0xEF]);
    
    let value_u16 = buf.get_u16();
    assert_eq!(buf.chunk(), &[0xEF]);
    
    // For more complex parsing, inspect chunk first
    // then decide how to parse
    let mut data2 = Bytes::from(vec![0x01, 0x02, 0x03, 0x04, 0x05]);
    
    while data2.has_remaining() {
        let chunk = data2.chunk();
        
        // Inspect the next byte
        if chunk[0] == 0x01 {
            // Read as u32
            let value = data2.get_u32();
            println!("Read as u32: {:08X}", value);
        } else {
            // Read as u8
            let value = data2.get_u8();
            println!("Read as u8: {:02X}", value);
        }
    }
}

Use chunk() to inspect upcoming data before deciding how to parse it with get_* methods.

Zero-Copy Parsing Patterns

use bytes::{Bytes, Buf};
use std::io::{self, Write};
 
fn main() -> io::Result<()> {
    let data = Bytes::from(b"HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nHello".as_slice());
    
    // Parse HTTP response without copying
    let mut buf = data;
    
    // Find the header/body separator
    let header_end = find_header_end(&buf);
    if let Some(end) = header_end {
        // Access header chunk without copying
        let header_chunk = &buf.chunk()[..end];
        println!("Headers: {}", std::str::from_utf8(header_chunk).unwrap());
        
        // Advance past headers
        buf.advance(end + 4);  // +4 for \r\n\r\n
        
        // Now chunk() points to body
        let body_chunk = buf.chunk();
        println!("Body: {}", std::str::from_utf8(body_chunk).unwrap());
    }
    
    // Write buffer directly to output without copying
    let data = Bytes::from("Direct write without copy");
    io::stdout().write_all(data.chunk())?;
    
    Ok(())
}
 
fn find_header_end<B: Buf>(buf: &B) -> Option<usize> {
    let chunk = buf.chunk();
    
    // Look for \r\n\r\n
    for i in 0..chunk.len().saturating_sub(3) {
        if &chunk[i..i+4] == b"\r\n\r\n" {
            return Some(i);
        }
    }
    
    None
}

chunk() enables zero-copy parsing by allowing inspection without advancing or copying.

Working with Take for Limited Views

use bytes::{Bytes, Buf};
use bytes::buf::Take;
 
fn main() {
    let data = Bytes::from("hello world this is a test");
    
    // Take creates a limited view of a buffer
    let mut limited: Take<Bytes> = data.take(11);  // "hello world"
    
    // chunk() returns up to the limit
    assert_eq!(limited.chunk(), b"hello world");
    assert_eq!(limited.remaining(), 11);
    
    // advance() respects the limit
    limited.advance(6);
    assert_eq!(limited.chunk(), b"world");
    assert_eq!(limited.remaining(), 5);
    
    // The original buffer is unchanged
    let original = limited.into_inner();
    assert_eq!(original.chunk(), b"hello world this is a test");
    
    // Use Take to process a fixed-size header
    let data = Bytes::from("HEADER12body data here");
    
    // Take first 8 bytes as header
    let mut header = data.clone().take(8);
    let header_data = header.chunk();
    println!("Header: {:?}", std::str::from_utf8(header_data));
    
    // Advance into the original to get body
    let mut body = data.clone();
    body.advance(8);
    println!("Body: {:?}", std::str::from_utf8(body.chunk()));
}

Take limits how much data chunk() returns, useful for fixed-size headers or length-prefixed data.

Custom Buf Implementations

use bytes::Buf;
 
// A simple buffer that wraps a slice
struct SliceBuf<'a> {
    data: &'a [u8],
    pos: usize,
}
 
impl<'a> SliceBuf<'a> {
    fn new(data: &'a [u8]) -> Self {
        Self { data, pos: 0 }
    }
}
 
impl<'a> Buf for SliceBuf<'a> {
    fn remaining(&self) -> usize {
        self.data.len() - self.pos
    }
    
    fn chunk(&self) -> &[u8] {
        &self.data[self.pos..]
    }
    
    fn advance(&mut self, cnt: usize) {
        assert!(cnt <= self.remaining());
        self.pos += cnt;
    }
}
 
fn main() {
    let data = b"hello world";
    let mut buf = SliceBuf::new(data);
    
    // chunk() returns remaining data from current position
    assert_eq!(buf.chunk(), b"hello world");
    
    buf.advance(6);
    assert_eq!(buf.chunk(), b"world");
    
    // Process all data
    buf.advance(buf.remaining());
    assert_eq!(buf.chunk(), b"");
    assert!(!buf.has_remaining());
}

Implement Buf for custom types by providing remaining(), chunk(), and advance().

Performance Implications

use bytes::{Bytes, Buf};
use std::time::Instant;
 
fn main() {
    // Create a large buffer
    let size = 10_000_000;
    let data = Bytes::from(vec![0u8; size]);
    
    // Using chunk() - no copy
    let start = Instant::now();
    let chunk = data.chunk();
    let _sum: u64 = chunk.iter().map(|&b| b as u64).sum();
    let chunk_duration = start.elapsed();
    println!("chunk() processing: {:?}", chunk_duration);
    
    // Using copy_to_bytes() - allocates and copies
    let mut data2 = Bytes::from(vec![0u8; size]);
    let start = Instant::now();
    let copied = data2.copy_to_bytes(size);
    let _sum: u64 = copied.chunk().iter().map(|&b| b as u64).sum();
    let copy_duration = start.elapsed();
    println!("copy_to_bytes() processing: {:?}", copy_duration);
    
    // chunk() is much faster because it doesn't allocate
    // The difference is more pronounced with larger buffers
    
    // For small buffers, the difference is negligible
    let small_data = Bytes::from("hello");
    let _chunk = small_data.chunk();  // Fast, no copy
    // Even copy_to_bytes(5) would be fast for small sizes
}

chunk() avoids allocation and copying; copy_to_bytes() pays O(n) cost for allocation and copy.

Synthesis

Method comparison:

| Method | Returns | Copies | Advances | |--------|---------|--------|----------| | chunk() | &[u8] | No | No | | copy_to_bytes(n) | Bytes | Yes | Yes | | copy_to_slice(dst) | () | Yes | Yes | | to_bytes() | Bytes | Maybe | No |

chunk behavior by buffer type:

| Buffer Type | chunk() Returns | |-------------|-----------------| | Bytes | All remaining bytes | | BytesMut | All remaining bytes | | Chain<A, B> | First buffer's remaining bytes | | Take<B> | Up to limit bytes | | Cursor<T> | All remaining bytes | | Custom Buf | Implementor-defined |

Common patterns:

// Inspect before consuming
if buf.chunk().starts_with(b"HTTP/") {
    // Parse as HTTP
}
 
// Zero-copy write to output
output.write_all(buf.chunk())?;
 
// Process multiple chunks
while buf.has_remaining() {
    let chunk = buf.chunk();
    process(chunk);
    buf.advance(chunk.len());
}
 
// Parse fixed-size header
let header = buf.chunk().get(..8)?;
buf.advance(8);

Key insight: chunk() is the foundation of zero-copy buffer processing in the bytes crate. It provides a window into the buffer's internal storage without any allocation or data movement—just pointer arithmetic. This is essential for high-performance I/O where buffer copies would otherwise dominate CPU usage. For simple contiguous buffers like Bytes, chunk() returns everything; for composite buffers like Chain, it returns only the first segment. The combination of chunk() for inspection and advance() for cursor movement lets you traverse any buffer type uniformly, whether it's a single contiguous block or a chain of multiple fragments. This abstraction enables efficient parsing of protocols like HTTP or TLS, where you might receive data in multiple chunks but want to process it as a logical stream without ever copying the data into a single contiguous buffer. The alternative methods—copy_to_bytes(), to_bytes()—have their place when you need owned data to pass across thread boundaries or store for later use, but chunk() should be your default choice when you just need to read or process the current buffer contents.

Loading page…

What is the purpose of bytes::Buf::chunk for accessing underlying buffer slices without copying?

Basic chunk Usage

chunk with Buffer Cursor and advance

chunk with BytesMut for Mutation

Composite Buffers with Chain

Iterating Over All Chunks

Comparison with copy_to_bytes

Reading Typed Data with chunk

Zero-Copy Parsing Patterns

Working with Take for Limited Views

Custom Buf Implementations

Performance Implications

Synthesis

What is the purpose of bytes::Buf::chunk for accessing underlying buffer slices without copying?

Basic chunk Usage

chunk with Buffer Cursor and advance

chunk with BytesMut for Mutation

Composite Buffers with Chain

Iterating Over All Chunks

Comparison with copy_to_bytes

Reading Typed Data with chunk

Zero-Copy Parsing Patterns

Working with Take for Limited Views

Custom Buf Implementations

Performance Implications

Synthesis

What is the purpose of `bytes::Buf::chunk` for accessing underlying buffer slices without copying?

What is the purpose of `bytes::Buf::chunk` for accessing underlying buffer slices without copying?