How does `bytes::Buf::chunk` differ from `bytes` for accessing underlying data slices without copying?

Buf::chunk returns a slice reference to a contiguous portion of the buffer's contents without copying, allowing callers to read data directly from the underlying storage—whether that's a single Bytes reference, a Vec, or a chain of buffers. Unlike bytes() which always returns a Bytes clone (potentially with reference counting overhead), chunk() returns &[u8] which is a direct view into whatever storage backs the buffer. For contiguous buffers like Bytes or Vec, chunk() provides the entire contents; for chains, it provides just the first contiguous segment. This enables efficient parsing where you read header bytes from chunk(), advance past them, and repeat—without allocating or copying.

The Buf Trait and chunk Method

use bytes::Buf;
 
// The Buf trait provides the chunk method:
pub trait Buf {
    fn chunk(&self) -> &[u8];
    // ... other methods
}
 
// chunk returns a slice reference to contiguous data
// No allocation, no copy, just a reference

chunk() returns a reference to contiguous bytes in the buffer.

Basic chunk Usage

use bytes::{Buf, Bytes};
 
fn basic_chunk() {
    let bytes = Bytes::from("hello world");
    
    // chunk returns a reference to all data
    let slice: &[u8] = bytes.chunk();
    
    // No copy happened - just a reference
    assert_eq!(slice, b"hello world");
    assert_eq!(slice.len(), 11);
    
    // The original Bytes still owns the data
    // slice is a borrowed reference
}
 
fn with_vec() {
    let vec = vec
![1u8, 2, 3, 4, 5];
    
    // Vec implements Buf
    let slice: &[u8] = vec.chunk();
    
    // Direct reference to Vec's data
    assert_eq!(slice, &[1, 2, 3, 4, 5]);
}

chunk() returns &[u8] - a borrowed slice, not an owned type.

chunk vs bytes() Method

use bytes::{Buf, Bytes};
 
fn chunk_vs_bytes() {
    let original = Bytes::from("hello world");
    
    // chunk(): returns &[u8] - borrowed reference
    let borrowed: &[u8] = original.chunk();
    // No allocation, no reference count increment
    
    // Bytes::bytes() doesn't exist - I need to check
    
    // Actually, the question might be about:
    // - chunk() vs copying to Vec
    // - chunk() vs Bytes::copy_to_bytes()
    
    let original = Bytes::from("hello world");
    
    // copy_to_bytes(): creates new Bytes (reference counted copy)
    let owned: Bytes = original.copy_to_bytes(5);
    // This shares the underlying storage (cheap)
    // But is still a Bytes handle, not &[u8]
    
    // chunk(): reference to contiguous bytes
    let borrowed: &[u8] = original.chunk();
    // Pure reference, no handle creation
}

chunk() returns a borrowed slice; copy_to_bytes() returns a new Bytes handle.

Zero-Copy Access Pattern

use bytes::{Buf, Bytes};
 
fn zero_copy_reading() {
    let mut buf = Bytes::from("hello world");
    
    // Read first 5 bytes without copying
    let first_five: &[u8] = &buf.chunk()[..5];
    assert_eq!(first_five, b"hello");
    
    // Advance past those bytes
    buf.advance(5);
    
    // Now chunk returns remaining data
    let remaining: &[u8] = buf.chunk();
    assert_eq!(remaining, b" world");
    
    // Total copies: 0
    // Total allocations: 0
}

The pattern: chunk() to view, advance() to skip, repeat.

Parsing with chunk

use bytes::{Buf, Bytes};
 
fn parse_header(buf: &mut Bytes) -> Option<u8> {
    // Need at least 2 bytes for header
    if buf.remaining() < 2 {
        return None;
    }
    
    let chunk = buf.chunk();
    
    // Read directly from chunk - no copy
    let version = chunk[0];
    let length = chunk[1] as usize;
    
    // Validate length before advancing
    if buf.remaining() < 2 + length {
        return None;
    }
    
    // Advance past header
    buf.advance(2);
    
    Some(version)
}
 
fn parse_example() {
    let mut data = Bytes::from(&[0x01, 0x05, 1, 2, 3, 4, 5][..]);
    
    let version = parse_header(&mut data).unwrap();
    assert_eq!(version, 0x01);
    assert_eq!(data.remaining(), 5);
}

Parse headers directly from chunk() without intermediate copies.

chunk with Buf::copy_to_bytes

use bytes::{Buf, Bytes};
 
fn chunk_vs_copy_to_bytes() {
    let mut buf = Bytes::from("hello world");
    
    // chunk(): borrowed slice
    let borrowed: &[u8] = buf.chunk();
    
    // copy_to_bytes(n): owned Bytes handle
    // Takes n bytes from the front
    let owned: Bytes = buf.copy_to_bytes(5);
    
    // Key differences:
    // - chunk() doesn't consume from buffer
    // - copy_to_bytes() advances the buffer
    
    // After copy_to_bytes:
    assert_eq!(buf.remaining(), 6);  // " world"
    
    // After chunk (no change):
    // buf is unchanged
}

chunk() peeks; copy_to_bytes() consumes and returns an owned handle.

Contiguous vs Non-Contiguous Buffers

use bytes::{Buf, Bytes, BytesMut};
 
fn contiguous_buffer() {
    // Bytes is contiguous - single slice
    let bytes = Bytes::from("hello");
    let chunk = bytes.chunk();
    // chunk.len() == bytes.remaining()
    assert_eq!(chunk.len(), bytes.remaining());
    
    // Vec is contiguous
    let vec = vec
![1u8, 2, 3];
    let chunk = vec.chunk();
    assert_eq!(chunk.len(), vec.remaining());
}
 
fn chain_non_contiguous() {
    use bytes::Buf;
    
    // Chain is non-contiguous - multiple slices
    let buf1 = Bytes::from("hello");
    let buf2 = Bytes::from(" ");
    let buf3 = Bytes::from("world");
    
    let chain = buf1.chain(buf2).chain(buf3);
    
    // chunk() returns only FIRST contiguous segment
    let chunk = chain.chunk();
    assert_eq!(chunk, b"hello");  // Only first segment!
    
    // To see remaining segments, need to iterate chunks
    // or use advance to move through
}

For chained buffers, chunk() returns only the first contiguous segment.

Iterating Chunks

use bytes::{Buf, Bytes};
 
fn iterate_chunks() {
    let buf1 = Bytes::from("hello");
    let buf2 = Bytes::from(" ");
    let buf3 = Bytes::from("world");
    let mut chain = buf1.chain(buf2).chain(buf3);
    
    // Process all chunks without copying
    let mut total_len = 0;
    while chain.has_remaining() {
        let chunk = chain.chunk();
        total_len += chunk.len();
        // Process chunk without copy
        chain.advance(chunk.len());
    }
    
    assert_eq!(total_len, 11);  // "hello world".len()
}

Loop through chunk() and advance() to process all data without copying.

Bytes::chunk Implementation

use bytes::Bytes;
 
fn bytes_chunk_implementation() {
    // Bytes stores reference-counted shared buffer
    // chunk() returns slice into that buffer
    
    let bytes = Bytes::from("hello world");
    
    // chunk() returns &[u8] pointing into Bytes' storage
    let chunk = bytes.chunk();
    
    // This is essentially free:
    // - No reference count increment (we're borrowing)
    // - No allocation
    // - Just pointer arithmetic
    
    // Multiple slices can reference same underlying storage
    let bytes2 = bytes.clone();  // Increments ref count
    let chunk1 = bytes.chunk();
    let chunk2 = bytes2.chunk();
    // Both point to same data
}

Bytes::chunk() returns a slice into the reference-counted storage.

BytesMut::chunk Implementation

use bytes::BytesMut;
 
fn bytes_mut_chunk() {
    let mut buf = BytesMut::with_capacity(100);
    buf.extend_from_slice(b"hello");
    
    // chunk returns slice into buffer's internal Vec
    let chunk = buf.chunk();
    assert_eq!(chunk, b"hello");
    
    // Modifications affect the chunk view
    // (but you can't modify while chunk is borrowed)
    
    // After modification:
    buf.extend_from_slice(b" world");
    let chunk = buf.chunk();
    assert_eq!(chunk, b"hello world");
}

BytesMut::chunk() returns a slice into the internal Vec.

When chunk Returns Partial Data

use bytes::{Buf, Bytes};
 
fn partial_chunks() {
    // Single contiguous Bytes: chunk returns everything
    let bytes = Bytes::from("hello");
    assert_eq!(bytes.chunk().len(), 5);
    
    // After advance: chunk returns remaining
    let mut bytes = Bytes::from("hello world");
    bytes.advance(6);
    assert_eq!(bytes.chunk(), b"world");
    
    // Chain: chunk returns first segment
    let a = Bytes::from("a");
    let b = Bytes::from("b");
    let mut chain = a.chain(b);
    
    assert_eq!(chain.chunk(), b"a");  // Only first!
    chain.advance(1);
    assert_eq!(chain.chunk(), b"b");  // Now second
}

chunk() returns contiguous data from current position to end of current segment.

Performance: Zero Copy Parsing

use bytes::{Buf, Bytes};
 
fn zero_copy_parser() {
    // Parse a simple protocol:
    // [length: u16][payload: bytes]
    
    fn parse_message(buf: &mut Bytes) -> Option<Bytes> {
        if buf.remaining() < 2 {
            return None;
        }
        
        // Peek at length without copying
        let len = u16::from_be_bytes([
            buf.chunk()[0],
            buf.chunk()[1],
        ]);
        
        if buf.remaining() < 2 + len as usize {
            return None;
        }
        
        // Skip length field
        buf.advance(2);
        
        // Extract payload as Bytes (reference counted, not copied)
        let payload = buf.copy_to_bytes(len as usize);
        
        Some(payload)
    }
    
    // No copies made during parsing
    // Only Bytes handles created (cheap reference counting)
}

Parse protocols by peeking with chunk(), then consuming with advance().

chunk vs to_bytes

use bytes::{Buf, Bytes};
 
fn chunk_vs_to_bytes() {
    let mut buf = Bytes::from("hello");
    
    // chunk(): &[u8] - borrowed reference
    let borrowed: &[u8] = buf.chunk();
    
    // copy_to_bytes(n): Bytes - owned handle
    let owned: Bytes = buf.copy_to_bytes(5);
    
    // to_bytes() would convert all remaining (if existed)
    // But Buf trait doesn't have to_bytes()
    // BytesMut has into_bytes() which consumes entirely
    
    // The difference:
    // - borrowed: lifetime tied to source
    // - owned: independent handle (via reference counting)
}

Borrowed slices have limited lifetimes; owned handles are independent.

Memory Layout with Bytes

use bytes::Bytes;
 
fn memory_layout() {
    // Bytes is a smart pointer over shared data
    // 
    // Structure:
    // - Pointer to data
    // - Length of slice
    // - Reference to shared Arc (for reference counting)
    
    let original = Bytes::from("hello world");
    
    // chunk() returns a &[u8] that points into that data
    // &[u8] is: { pointer, length }
    // pointer = original's data pointer
    // length = original's length
    
    // This is why chunk() is zero-cost:
    // - No allocation (just stack struct)
    // - No copy (references same memory)
    // - No ref count bump (borrowing, not cloning)
}

chunk() returns a fat pointer (pointer + length) into existing storage.

Combining chunk with Buf Methods

use bytes::{Buf, Bytes, BytesMut};
 
fn combined_operations() {
    let mut buf = BytesMut::new();
    buf.extend_from_slice(&[1, 2, 3, 4, 5, 6, 7, 8]);
    
    // Check bytes without consuming
    let first_byte = buf.chunk()[0];
    assert_eq!(first_byte, 1);
    buf.remaining();  // Still 8
    
    // Read u16 (advances buffer)
    let val = buf.get_u16();  // Uses chunk internally
    assert_eq!(val, 0x0102);
    assert_eq!(buf.remaining(), 6);
    
    // chunk now reflects remaining data
    assert_eq!(buf.chunk(), &[3, 4, 5, 6, 7, 8]);
    
    // The Buf methods use chunk() internally for efficiency:
    // - get_u8, get_u16, etc. read from chunk then advance
    // - No intermediate copies
}

Buf methods like get_u16() internally use chunk() for efficiency.

Real-World: HTTP Parsing

use bytes::{Buf, BytesMut};
 
fn http_parsing() {
    // Simplified HTTP request line parsing
    // "GET /path HTTP/1.1\r\n"
    
    fn parse_request_line(buf: &mut BytesMut) -> Option<(&[u8], &[u8])> {
        // Peek at all available data
        let data = buf.chunk();
        
        // Find end of line
        let line_end = data.windows(2).position(|w| w == b"\r\n")?;
        
        // Extract method and path without copying
        let line = &data[..line_end];
        let space1 = line.iter().position(|&b| b == b' ')?;
        let space2 = line.iter().rposition(|&b| b == b' ')?;
        
        let method = &line[..space1];
        let path = &line[space1 + 1..space2];
        
        // Consume the line
        buf.advance(line_end + 2);
        
        Some((method, path))
    }
    
    let mut buf = BytesMut::from("GET /path HTTP/1.1\r\nHost: example.com");
    
    if let Some((method, path)) = parse_request_line(&mut buf) {
        assert_eq!(method, b"GET");
        assert_eq!(path, b"/path");
    }
}

HTTP parsing can use chunk() to peek at headers without copying.

When to Use chunk vs copy_to_bytes

use bytes::{Buf, Bytes};
 
fn when_to_use_which() {
    // Use chunk() when:
    // - You only need to read (not consume)
    // - You need a slice for an API
    // - You want to peek ahead
    // - Zero-copy is important
    
    // Use copy_to_bytes() when:
    // - You need an owned Bytes handle
    // - You want to pass data to another task
    // - You need 'static lifetime
    // - You're extracting a portion to keep
    
    let mut buf = Bytes::from("hello world");
    
    // Peek: use chunk
    let peek = &buf.chunk()[..5];
    assert_eq!(peek, b"hello");
    // buf unchanged
    
    // Extract: use copy_to_bytes
    let extracted = buf.copy_to_bytes(5);
    // buf advanced, extracted is independent
}

Use chunk() for peeking, copy_to_bytes() for extracting owned data.

Working with Vec

use bytes::Buf;
 
fn vec_chunk() {
    let vec = vec
![1u8, 2, 3, 4, 5];
    
    // Vec<u8> implements Buf
    // chunk returns slice into Vec
    
    let slice: &[u8] = vec.chunk();
    assert_eq!(slice, &[1, 2, 3, 4, 5]);
    
    // This is essentially free:
    // slice is &vec[..]
    
    // Unlike Bytes, Vec doesn't have reference counting
    // slice lifetime is tied to Vec
}

Vec<u8> implements Buf, and chunk() returns a slice into the Vec.

Synthesis

What chunk() provides:

// chunk() returns &[u8]:
// - Borrowed slice into buffer's storage
// - No allocation
// - No copy
// - Lifetime tied to buffer
 
// For contiguous buffers (Bytes, Vec, BytesMut):
// - chunk() returns all remaining data
 
// For non-contiguous buffers (Chain):
// - chunk() returns first contiguous segment
// - Must iterate to see all data

Zero-copy pattern:

// Typical usage pattern:
let mut buf = /* some Buf implementation */;
 
// 1. Peek at data
let data = buf.chunk();
 
// 2. Parse from slice (no copy)
let length = u16::from_be_bytes([data[0], data[1]]);
 
// 3. Consume parsed bytes
buf.advance(2);
 
// 4. Repeat

Key differences:

// chunk(): borrowed &[u8]
// - Zero cost
// - Lifetime bound to source
// - Read-only
 
// copy_to_bytes(n): owned Bytes
// - Reference count bump (cheap)
// - Independent handle
// - Can outlive source
// - Consumes from buffer
 
// to_vec() / to_bytes(): owned collection
// - Allocation + copy
// - Independent lifetime
// - Expensive for large data

When chunk() is partial:

// For Chain<T, U>:
// chunk() only returns first segment
// To process all: loop chunk() + advance()
 
let mut chain = a.chain(b);
while chain.has_remaining() {
    process(chain.chunk());  // Zero copy
    chain.advance(chunk.len());
}

Key insight: Buf::chunk() provides direct, zero-copy access to buffer contents as a &[u8] slice, enabling efficient parsing without allocation. For contiguous buffers like Bytes or Vec, chunk() returns all remaining data; for chains, it returns the first contiguous segment. This differs from methods like copy_to_bytes() which create new handles (with reference counting overhead) or to_vec() which allocate and copy. The primary use case is parsing: peek at chunk(), read what you need, advance() past it, and repeat—all without allocating intermediate buffers or copying data.

How does bytes::Buf::chunk differ from bytes for accessing underlying data slices without copying?