What are the trade-offs between smallvec::SmallVec::from_buf and from_buf_and_len for initializing from raw storage?
from_buf assumes the provided buffer is empty (zero initialized length), while from_buf_and_len accepts an explicit length parameter for buffers that already contain initialized elements. The critical distinction is safety: from_buf is safe because it treats all elements as uninitialized, while from_buf_and_len is unsafe because it trusts the caller to provide a valid length for already-initialized elements.
SmallVec Overview: Inline and Heap Storage
use smallvec::SmallVec;
// SmallVec stores small arrays inline, spilling to heap when needed
// Array<[T; N]> stores up to N elements inline
type SmallIntVec = SmallVec<[i32; 4]>;
fn smallvec_basics() {
let mut v: SmallIntVec = SmallVec::new();
// First 4 elements stored inline (no heap allocation)
v.push(1);
v.push(2);
v.push(3);
v.push(4);
// 5th element spills to heap
v.push(5);
// The inline buffer is part of the struct:
// SmallVec { data: [MaybeUninit<T>; N], len: usize, ... }
}SmallVec uses an inline array of MaybeUninit<T> to store elements, avoiding heap allocation for small counts.
The from_buf Method: Empty Buffer Initialization
use smallvec::SmallVec;
fn from_buf_example() {
// from_buf takes an array and creates an empty SmallVec
// All elements are treated as uninitialized
let buf = [0i32; 4]; // Array of 4 zeros
let v: SmallVec<[i32; 4]> = SmallVec::from_buf(buf);
// The SmallVec is empty (len = 0)
assert_eq!(v.len(), 0);
assert!(v.is_empty());
// The buffer values (zeros) are irrelevant
// from_buf ignores the content, treating it as uninitialized
// This is SAFE because:
// - The length is known to be 0
// - No elements are read
// - The buffer is just storage space
}from_buf is safe because it always sets length to zero, never reading the provided buffer values.
The from_buf_and_len Method: Pre-initialized Buffer
use smallvec::SmallVec;
fn from_buf_and_len_example() {
// from_buf_and_len takes a buffer AND a length
// The first `len` elements must be initialized
let buf = [1i32, 2, 3, 4]; // Array with initialized values
let v: SmallVec<[i32; 4]> = unsafe {
SmallVec::from_buf_and_len(buf, 3) // First 3 elements are valid
};
// The SmallVec has length 3
assert_eq!(v.len(), 3);
assert_eq!(&v[..], &[1, 2, 3]);
// The 4th element (buf[3]) is now considered uninitialized
// Even though buf[3] = 4 in the original array
// This is UNSAFE because:
// - The caller claims 3 elements are initialized
// - If wrong, undefined behavior occurs
// - Rust can't verify the length is correct
}from_buf_and_len is unsafe because it trusts the caller's length claim for already-initialized elements.
Safety Analysis: Why from_buf_and_len is Unsafe
use smallvec::SmallVec;
fn safety_analysis() {
// UNSAFE: Claiming more elements than initialized
let buf = [1i32, 2, 0, 0]; // Only first 2 are meaningful
// WRONG - claiming 4 initialized elements:
// let v: SmallVec<[i32; 4]> = unsafe {
// SmallVec::from_buf_and_len(buf, 4) // UNDEFINED BEHAVIOR!
// };
// Why this is UB:
// - buf[2] and buf[3] contain 0, but are claimed as initialized
// - The values happen to be valid i32, but could be anything
// - If T has Drop, dropping uninitialized = UB
// - If T is read, arbitrary values appear
// SAFE: Correct length claim
let v: SmallVec<[i32; 4]> = unsafe {
SmallVec::from_buf_and_len(buf, 2) // Correct: only first 2 are initialized
};
assert_eq!(&v[..], &[1, 2]);
}The unsafety comes from claiming more elements than are actually initializedāthis is undefined behavior.
Use Case: Building from Computed Data
use smallvec::SmallVec;
fn computed_data() {
// from_buf_and_len is useful when you compute values into an array
fn compute_into_buffer() -> SmallVec<[f64; 8]> {
let mut buf: [f64; 8] = [0.0; 8];
let mut len = 0;
// Compute values
for i in 0..5 {
buf[i] = (i as f64).sqrt();
len += 1;
}
// SAFETY: We initialized exactly `len` elements
unsafe {
SmallVec::from_buf_and_len(buf, len)
}
}
let v = compute_into_buffer();
assert_eq!(v.len(), 5);
}from_buf_and_len is appropriate when you've computed values into a buffer and know exactly how many are valid.
Use Case: from_buf for Allocation Without Initialization
use smallvec::SmallVec;
fn from_buf_allocation() {
// from_buf is for when you want an empty SmallVec
// but have a buffer array you want to use as storage
// Common pattern: allocate inline buffer, fill later
let v: SmallVec<[String; 4]> = SmallVec::from_buf([
String::new(),
String::new(),
String::new(),
String::new(),
]);
// Wait - this doesn't make sense!
// We're creating 4 empty Strings just to discard them
// Better: use SmallVec::new() for empty SmallVec
let v: SmallVec<[String; 4]> = SmallVec::new();
// from_buf is useful when:
// 1. You have a zero-cost-to-create array (Copy types)
// 2. You want to reserve capacity inline
let v: SmallVec<[i32; 16]> = SmallVec::from_buf([0; 16]);
// Capacity is 16, length is 0
assert_eq!(v.len(), 0);
assert_eq!(v.capacity(), 16);
}from_buf is useful for pre-allocating inline capacity with Copy types, avoiding heap allocation.
Copy Types vs Drop Types
use smallvec::SmallVec;
fn copy_vs_drop_types() {
// Copy types: Safe to create uninitialized buffer
let buf: [i32; 4] = [0; 4]; // Cheap to create
let v: SmallVec<[i32; 4]> = SmallVec::from_buf(buf);
// Even though we set values, they're treated as uninit
// No Drop means no UB if we "forget" to use them
// Drop types: Must be careful
// let buf: [String; 4] = [String::new(); 4];
// let v: SmallVec<[String; 4]> = SmallVec::from_buf(buf);
// BAD: We allocated 4 Strings then treated them as uninit
// This would leak the 4 empty Strings!
// For Drop types, use new() or with_capacity():
let v: SmallVec<[String; 4]> = SmallVec::new(); // Safe
}For Copy types, from_buf([T::default(); N]) is cheap; for Drop types, it can cause resource leaks.
The from_buf_and_len_unchecked Variant
use smallvec::SmallVec;
fn unchecked_variant() {
let buf = [1i32, 2, 3, 4];
// from_buf_and_len_unugged is even more unsafe
// It doesn't check:
// 1. That len <= capacity
// 2. That the buffer is properly aligned
// SAFETY: Must ensure len <= buf.len() and all elements are initialized
let v: SmallVec<[i32; 4]> = unsafe {
SmallVec::from_buf_and_len_unchecked(buf, 4)
};
// from_buf_and_len is slightly safer:
// - It checks that len <= capacity
// - Still unsafe for element initialization
// Prefer from_buf_and_len over _unchecked unless you're certain
}from_buf_and_len_unchecked skips capacity checks; prefer from_buf_and_len for the minimal safety check.
Practical Example: Parsing into SmallVec
use smallvec::SmallVec;
fn parse_into_smallvec() {
// Scenario: Parsing a known-maximum number of values
fn parse_numbers(input: &[u8], max_count: usize) -> SmallVec<[u32; 16]> {
let mut buf: [u32; 16] = [0; 16];
let mut count = 0;
// Parse values into buffer
for (i, chunk) in input.split(|&b| b == b',').enumerate() {
if i >= 16 || count >= max_count {
break;
}
if let Ok(n) = std::str::from_utf8(chunk).unwrap().parse::<u32>() {
buf[count] = n;
count += 1;
}
}
// SAFETY: We initialized exactly `count` elements
unsafe {
SmallVec::from_buf_and_len(buf, count)
}
}
let result = parse_numbers(b"1,2,3,4,5", 16);
assert_eq!(&result[..], &[1, 2, 3, 4, 5]);
}This pattern avoids heap allocation while safely building a SmallVec from computed data.
Comparison: from_buf vs from_buf_and_len
use smallvec::SmallVec;
fn comparison() {
// | Aspect | from_buf | from_buf_and_len |
// |--------|----------|------------------|
// | Safety | Safe | Unsafe |
// | Length | Always 0 | Caller-specified |
// | Content | Treated as uninit | First `len` are valid |
// | Use case | Empty SmallVec with inline capacity | Pre-filled buffer |
let buf = [1, 2, 3, 4];
// from_buf: Safe, creates empty SmallVec
let v1: SmallVec<[i32; 4]> = SmallVec::from_buf(buf);
assert_eq!(v1.len(), 0); // Empty!
assert_eq!(v1.capacity(), 4); // But has capacity
// from_buf_and_len: Unsafe, creates populated SmallVec
let v2: SmallVec<[i32; 4]> = unsafe {
SmallVec::from_buf_and_len(buf, 4)
};
assert_eq!(v2.len(), 4); // Contains values
assert_eq!(&v2[..], &[1, 2, 3, 4]);
}from_buf creates empty; from_buf_and_len creates populated with trusted length.
Drop Behavior and Memory Safety
use smallvec::SmallVec;
fn drop_behavior() {
// When a SmallVec is dropped:
// 1. All elements with index < len are dropped
// 2. Inline storage is deallocated (part of struct)
// 3. Heap storage (if any) is deallocated
// from_buf:
// - len = 0
// - Nothing dropped (no elements claimed)
// from_buf_and_len(buf, len):
// - len = caller-provided
// - First `len` elements dropped
// - If len > actual initialized: UB on drop
// Example of UB scenario:
struct Data {
ptr: *mut i32,
}
impl Drop for Data {
fn drop(&mut self) {
unsafe { std::ptr::drop_in_place(self.ptr) };
}
}
// If we claim uninitialized Data elements:
// - Drop would read uninitialized pointer
// - Undefined behavior!
}If from_buf_and_len claims more elements than initialized, dropping causes undefined behavior.
Using with MaybeUninit
use smallvec::SmallVec;
use std::mem::MaybeUninit;
fn with_maybeuninit() {
// Working with MaybeUninit for proper initialization tracking
// Create uninitialized buffer
let mut uninit_buf: [MaybeUninit<i32>; 4] = MaybeUninit::uninit_array();
// Initialize first 2 elements
uninit_buf[0].write(1);
uninit_buf[1].write(2);
// Convert to initialized buffer
// Note: This is still unsafe because we must ensure proper initialization
let buf: [i32; 4] = unsafe {
// Transmute is tricky - the remaining 2 elements are still uninit
// This is NOT safe to do directly!
// Instead, use from_buf_and_len_raw (if available) or be careful
// Better approach:
std::mem::transmute_copy(&uninit_buf)
};
// Use from_buf_and_len with correct count
let v: SmallVec<[i32; 4]> = unsafe {
SmallVec::from_buf_and_len(buf, 2) // Only first 2 are valid
};
}Careful tracking of which elements are initialized is essential for sound from_buf_and_len usage.
Performance Considerations
use smallvec::SmallVec;
fn performance() {
// from_buf:
// - No runtime overhead for length tracking (always 0)
// - Buffer values are ignored (not read)
// - Good for Copy types where buffer creation is cheap
// from_buf_and_len:
// - Slight overhead for length check (len <= capacity)
// - from_buf_and_len_unchecked skips even this
// - Good when you've already computed values
// Alternative: SmallVec::new + push
let mut v: SmallVec<[i32; 4]> = SmallVec::new();
v.push(1);
v.push(2);
// Simple, safe, but may check capacity each push
// Alternative: SmallVec::from_slice
let v: SmallVec<[i32; 4]> = SmallVec::from_slice(&[1, 2, 3]);
// Safe, copies from slice
// May allocate if slice > capacity
}For most cases, prefer safe alternatives; use from_buf_and_len only when necessary for performance.
When to Use Each Method
use smallvec::SmallVec;
fn when_to_use() {
// Use from_buf when:
// 1. You need an empty SmallVec with inline capacity
// 2. You have a Copy type buffer available
// 3. You want to avoid Default::default() costs for heap types
// Example: Inline buffer for Copy type
let v: SmallVec<[u8; 256]> = SmallVec::from_buf([0u8; 256]);
// Use from_buf_and_len when:
// 1. You've initialized values into a buffer
// 2. You know exactly how many elements are valid
// 3. Performance-critical code avoiding copies
// Example: Pre-computed values
fn process_into_vec() -> SmallVec<[i32; 16]> {
let mut buf = [0i32; 16];
let mut count = 0;
// ... compute values into buf ...
for i in 0..5 {
buf[i] = i * 2;
count += 1;
}
unsafe { SmallVec::from_buf_and_len(buf, count) }
}
// DON'T use from_buf_and_len when:
// 1. Safe alternatives work (from_slice, push)
// 2. You're unsure about initialization count
// 3. The buffer contains Drop types you haven't initialized
}Use from_buf for empty vectors with pre-allocated inline storage; use from_buf_and_len when you have pre-initialized data.
Summary Table
use smallvec::SmallVec;
fn summary_table() {
// | Method | Safety | Result Length | Content |
// |--------|--------|---------------|---------|
// | from_buf | Safe | 0 | Empty |
// | from_buf_and_len | Unsafe | Caller-specified | First `len` elements |
// | from_buf_and_len_unchecked | More unsafe | Caller-specified | First `len` elements |
// | new | Safe | 0 | Empty |
// | from_slice | Safe | Slice length | Copy of slice |
// | When | Use |
// |------|-----|
// | Empty SmallVec, inline capacity | from_buf or new |
// | Pre-initialized data | from_buf_and_len (unsafe) |
// | Copy from slice | from_slice (safe) |
// | Performance-critical init | from_buf_and_len_unchecked (most unsafe) |
}Synthesis
Quick reference:
use smallvec::SmallVec;
fn quick_reference() {
// from_buf: Safe, creates empty SmallVec using provided buffer as storage
let empty: SmallVec<[i32; 4]> = SmallVec::from_buf([0; 4]);
assert_eq!(empty.len(), 0); // Always empty
// from_buf_and_len: Unsafe, creates populated SmallVec
let populated: SmallVec<[i32; 4]> = unsafe {
SmallVec::from_buf_and_len([1, 2, 3, 4], 3)
};
assert_eq!(&populated[..], &[1, 2, 3]); // First 3 elements
// Safety contract for from_buf_and_len:
// - Caller MUST ensure first `len` elements are properly initialized
// - If len > actual initialized: UNDEFINED BEHAVIOR
// - If len > capacity: panic or UB (depending on variant)
}Key insight: from_buf and from_buf_and_len differ in their assumptions about the buffer's contents and their safety guarantees. from_buf is safe because it treats the entire buffer as uninitialized storageāregardless of what values are in the array, the resulting SmallVec has length zero and will never read those values until elements are pushed. This makes it suitable for pre-allocating inline capacity with Copy types. from_buf_and_len is unsafe because it claims that the first len elements of the buffer are already initialized and validāthis is a trust relationship between the caller and the function that Rust cannot verify. If the caller lies about the length (claiming more elements than are actually initialized), the SmallVec will try to read, use, or drop values that may contain garbage data or invalid pointers, leading to undefined behavior. Use from_buf when you want an empty vector with inline capacity; use from_buf_and_len only when you have computed values into a buffer and can guarantee the exact count of initialized elements. For most cases, prefer safe alternatives like SmallVec::new() or SmallVec::from_slice() that don't require reasoning about initialization safety.