Arc<Mutex>

Guides

Rust walkthroughs

← Back to all guides

What are the trade-offs between reqwest::multipart::Part::bytes and file for upload payload construction?

Part::bytes loads the entire payload into memory before transmission, making it suitable for small data and in-memory content, while Part::file streams content directly from disk without fully loading it into memory, making it essential for large files and memory-constrained environments. The choice affects memory footprint, I/O patterns, error handling, and performance characteristics.

Basic Part::bytes Usage

use reqwest::multipart;
 
fn bytes_example() {
    // Create a part from in-memory bytes
    let data = b"Hello, World!".to_vec();
    let part = multipart::Part::bytes(data)
        .file_name("hello.txt")
        .mime_str("text/plain")
        .unwrap();
    
    // The bytes are already in memory
    // Transmission will read from this buffer
    
    let form = multipart::Form::new()
        .part("file", part);
    
    // Send the request
    let client = reqwest::Client::new();
    let response = client
        .post("https://example.com/upload")
        .multipart(form)
        .send();
}

Part::bytes takes ownership of a Vec<u8> containing the data.

Basic Part::file Usage

use reqwest::multipart;
use std::fs::File;
 
async fn file_example() -> Result<(), Box<dyn std::error::Error>> {
    // Create a part from a file path
    let part = multipart::Part::file("large_file.bin")
        .await?
        .file_name("large_file.bin")
        .mime_str("application/octet-stream")
        .unwrap();
    
    // The file is opened but NOT fully loaded into memory
    // Content is streamed during transmission
    
    let form = multipart::Form::new()
        .part("file", part);
    
    let client = reqwest::Client::new();
    let response = client
        .post("https://example.com/upload")
        .multipart(form)
        .send()
        .await?;
    
    Ok(())
}

Part::file opens the file and streams content during the request.

Memory Footprint Comparison

use reqwest::multipart;
 
fn memory_comparison() {
    // Part::bytes: Data stored in memory
    let large_data = vec
![0u8; 100 * 1024 * 1024]
;  // 100 MB
    let part = multipart::Part::bytes(large_data);
    // Memory usage: ~100 MB held in Vec
    
    // Part::file: File handle + buffer, not full content
    // Only a small buffer is kept in memory
    // Memory usage: ~8-64 KB depending on buffer size
    
    // For large files:
    // bytes approach: 1 GB file = 1 GB memory
    // file approach: 1 GB file = ~64 KB memory
    
    // Multiple concurrent uploads:
    // bytes approach: 10 x 100 MB uploads = 1 GB memory
    // file approach: 10 x 100 MB uploads = ~1 MB memory
}

Part::file dramatically reduces memory usage for large files.

Streaming Behavior

use reqwest::multipart;
use tokio::fs::File;
 
async fn streaming_behavior() {
    // Part::bytes: All data buffered before send
    // 1. Bytes loaded into Vec
    // 2. HTTP body created from Vec
    // 3. Transmission reads from Vec
    
    // Part::file: Data streamed during send
    // 1. File opened
    // 2. HTTP body created with file handle
    // 3. Transmission reads chunks from file
    
    // This matters for:
    // - Large files (don't fit in memory)
    // - Slow network (don't hold data in memory while waiting)
    // - Concurrent uploads (multiple files at once)
}

Part::file streams data during transmission; Part::bytes holds everything.

File Handle Management

use reqwest::multipart;
 
async fn file_handle_management() -> Result<(), Box<dyn std::error::Error>> {
    // Part::file manages file handle lifecycle
    let part = multipart::Part::file("document.pdf")
        .await?  // File opened here
        .file_name("document.pdf");
    
    // File handle is kept open until request completes
    // If request fails, handle is cleaned up properly
    
    let form = multipart::Form::new().part("file", part);
    
    // The file is read lazily during send()
    let client = reqwest::Client::new();
    let response = client
        .post("https://example.com/upload")
        .multipart(form)
        .send()
        .await?;
    
    // File handle closed after request completes
    
    Ok(())
}

Part::file handles file opening, reading, and closing automatically.

Async File Operations

use reqwest::multipart;
 
async fn async_file_upload() -> Result<(), Box<dyn std::error::Error>> {
    // Part::file is async (uses tokio::fs)
    let part = multipart::Part::file("data.bin").await?;
    
    // This uses async file I/O:
    // - Non-blocking file open
    // - Async reads during transmission
    // - Proper async cleanup
    
    // Part::bytes is sync (data already in memory)
    let data = tokio::fs::read("data.bin").await?;
    let part = multipart::Part::bytes(data);
    
    // With bytes:
    // - You do the async read manually
    // - reqwest just uses the in-memory buffer
    
    Ok(())
}

Part::file uses async I/O; Part::bytes requires you to handle file reading.

Error Handling Differences

use reqwest::multipart;
use std::io;
 
async fn error_handling() -> Result<(), Box<dyn std::error::Error>> {
    // Part::bytes: Errors happen immediately
    let data = vec
![1, 2, 3]
;
    let part = multipart::Part::bytes(data);
    // No I/O errors possible - data is already in memory
    
    // Part::file: Errors can occur at multiple points
    let part_result = multipart::Part::file("nonexistent.txt").await;
    match part_result {
        Ok(part) => { /* use part */ },
        Err(e) => {
            // File open failed
            eprintln!("Failed to open file: {}", e);
        }
    }
    
    // Errors during file read happen during send()
    let part = multipart::Part::file("data.bin").await?;
    let response = client.post(url).multipart(form).send().await;
    // If file read fails during transmission, send() returns error
    
    Ok(())
}

Part::file has more error points; Part::bytes errors are known upfront.

Content-Length and Headers

use reqwest::multipart;
use std::fs;
 
async fn content_length() -> Result<(), Box<dyn std::error::Error>> {
    // Part::bytes: Content-Length known immediately
    let data = vec
![1, 2, 3, 4, 5]
;
    let part = multipart::Part::bytes(data.clone());
    // Length = data.len() = 5
    
    // Part::file: Content-Length from file metadata
    let part = multipart::Part::file("data.bin").await?;
    // Length determined by file size
    // Can use Content-Length header for progress tracking
    
    // However, some servers require Content-Length
    // Part::file provides it from file metadata
    let metadata = fs::metadata("data.bin")?;
    let file_size = metadata.len();
    println!("Uploading {} bytes", file_size);
    
    Ok(())
}

Both approaches provide content length for proper HTTP headers.

Use Cases: When to Use bytes

use reqwest::multipart;
 
async fn when_to_use_bytes() {
    // Use Part::bytes when:
    
    // 1. Data is already in memory
    let json_data = serde_json::to_vec(&my_struct).unwrap();
    let part = multipart::Part::bytes(json_data)
        .file_name("data.json")
        .mime_str("application/json")
        .unwrap();
    
    // 2. Small data (fits comfortably in memory)
    let small_config = b"setting=value".to_vec();
    let part = multipart::Part::bytes(small_config);
    
    // 3. Generated content (not from file)
    let csv_content = generate_csv();  // Returns Vec<u8>
    let part = multipart::Part::bytes(csv_content)
        .file_name("report.csv");
    
    // 4. Need to modify data before upload
    let mut image = load_image();
    watermark_image(&mut image);
    let part = multipart::Part::bytes(image);
    
    // 5. Testing/mocking
    let test_data = b"test content".to_vec();
    let part = multipart::Part::bytes(test_data);
    
    // 6. Encrypted/encoded content
    let encrypted = encrypt(data);
    let part = multipart::Part::bytes(encrypted);
}

Use Part::bytes for in-memory, small, or processed data.

Use Cases: When to Use file

use reqwest::multipart;
 
async fn when_to_use_file() {
    // Use Part::file when:
    
    // 1. Large files (memory efficiency)
    let part = multipart::Part::file("video.mp4")
        .await
        .unwrap()
        .file_name("video.mp4");
    
    // 2. Memory-constrained environments
    let part = multipart::Part::file("large_dataset.csv")
        .await
        .unwrap();
    
    // 3. No need to process file content
    let part = multipart::Part::file("backup.tar.gz")
        .await
        .unwrap();
    
    // 4. Concurrent uploads of multiple files
    let files = vec
!["file1.bin", "file2.bin", "file3.bin"]
;
    let parts: Vec<_> = files.into_iter()
        .map(|f| multipart::Part::file(f))
        .collect();  // Won't load all into memory at once
    
    // 5. Server-side file uploads (already on disk)
    let part = multipart::Part::file("/var/data/upload/temp_file")
        .await
        .unwrap();
    
    // 6. Zero-copy uploads (stream from disk to network)
    let part = multipart::Part::file("/data/archive.zip")
        .await
        .unwrap();
}

Use Part::file for large files or memory efficiency.

Performance Comparison

use reqwest::multipart;
use std::time::Instant;
 
async fn performance_comparison() {
    // Scenario: 100 MB file upload
    
    // Part::bytes approach:
    // 1. Read entire file into memory: ~50-100ms
    // 2. Create Part: instant
    // 3. Send: network speed
    // Total memory: 100 MB held throughout
    
    let start = Instant::now();
    let data = tokio::fs::read("100MB.bin").await.unwrap();
    let part = multipart::Part::bytes(data);
    println!("bytes prep time: {:?}", start.elapsed());
    // Memory: 100 MB allocated
    
    // Part::file approach:
    // 1. Open file: ~1ms
    // 2. Create Part: instant
    // 3. Send: network speed, reads on-demand
    // Total memory: ~64 KB buffer
    
    let start = Instant::now();
    let part = multipart::Part::file("100MB.bin").await.unwrap();
    println!("file prep time: {:?}", start.elapsed());
    // Memory: ~64 KB allocated
    
    // Part::file is faster to start, uses less memory
}

Part::file has faster startup and lower memory usage.

Reading from Reader

use reqwest::multipart;
use std::io::Cursor;
 
async fn from_reader() {
    // Part::bytes with reader
    let data = b"content from reader".to_vec();
    let cursor = Cursor::new(data);
    let bytes: Vec<u8> = cursor.into_inner();
    let part = multipart::Part::bytes(bytes);
    
    // For streaming from a reader:
    // reader() method is available for custom streams
    let data = vec
![1, 2, 3, 4, 5]
;
    let cursor = Cursor::new(data);
    let part = multipart::Part::reader(cursor);
    
    // Part::reader streams from any Read implementation
    // But requires the reader to be 'static
    
    // Part::reader_with_length for known sizes
    let data = vec
![1, 2, 3, 4, 5]
;
    let cursor = Cursor::new(data.clone());
    let part = multipart::Part::reader_with_length(cursor, data.len() as u64);
}

Part::reader provides streaming for non-file sources.

Memory-Mapped Files

use reqwest::multipart;
 
async fn large_file_upload() {
    // For very large files, Part::file handles them efficiently
    // File size doesn't matter - it streams chunks
    
    // 10 GB file:
    let part = multipart::Part::file("huge_archive.tar")
        .await
        .unwrap();
    
    // Memory usage stays constant regardless of file size
    // The file is read in chunks during transmission
    
    // This is equivalent to memory-mapped file behavior
    // Without the complexity of actually memory-mapping
}

Part::file handles arbitrarily large files efficiently.

Combining Both Approaches

use reqwest::multipart;
 
async fn combined_upload() -> Result<(), Box<dyn std::error::Error>> {
    let form = multipart::Form::new()
        // Small metadata from memory
        .part("metadata", multipart::Part::bytes(
            serde_json::to_vec(&metadata)?
        )
            .file_name("metadata.json")
            .mime_str("application/json")
            .unwrap())
        // Large file from disk
        .part("data", multipart::Part::file("large_data.bin")
            .await?
            .file_name("data.bin")
            .mime_str("application/octet-stream")
            .unwrap())
        // Another small part
        .part("signature", multipart::Part::bytes(
            generate_signature()
        )
            .mime_str("application/octet-stream")
            .unwrap());
    
    let client = reqwest::Client::new();
    let response = client
        .post("https://example.com/upload")
        .multipart(form)
        .send()
        .await?;
    
    Ok(())
}

Mix bytes and file parts in the same form based on data characteristics.

Progress Tracking

use reqwest::multipart;
use std::fs;
 
async fn progress_tracking() -> Result<(), Box<dyn std::error::Error>> {
    // For Part::bytes: Total size known upfront
    let data = tokio::fs::read("file.bin").await?;
    let total_size = data.len();
    let part = multipart::Part::bytes(data);
    // Progress: bytes_sent / total_size
    
    // For Part::file: Size from metadata
    let metadata = tokio::fs::metadata("file.bin").await?;
    let total_size = metadata.len();
    let part = multipart::Part::file("file.bin").await?;
    // Progress: bytes_sent / total_size
    
    // Both allow progress tracking
    // Part::file must query file system for size
    
    Ok(())
}

Both support progress tracking; Part::file needs metadata query for size.

Concurrency Considerations

use reqwest::multipart;
 
async fn concurrent_uploads() {
    // Uploading 100 files of 10 MB each
    
    // With Part::bytes: 1 GB memory needed
    let files: Vec<Vec<u8>> = read_all_files().await;  // 1 GB in memory
    let parts: Vec<_> = files.into_iter()
        .map(multipart::Part::bytes)
        .collect();
    
    // With Part::file: ~6.4 MB memory (100 x 64 KB buffers)
    let files: Vec<&str> = get_file_paths();
    let parts: Vec<_> = files.into_iter()
        .map(|f| multipart::Part::file(f))
        .collect();
    
    // For concurrent uploads:
    // Part::file allows many simultaneous uploads
    // Part::bytes exhausts memory quickly
    
    // Example: Upload 10 files in parallel
    let handles: Vec<_> = file_paths.into_iter()
        .map(|path| {
            tokio::spawn(async move {
                let part = multipart::Part::file(path).await.unwrap();
                // upload...
            })
        })
        .collect();
}

Part::file enables concurrent uploads without memory exhaustion.

Security Considerations

use reqwest::multipart;
 
async fn security() {
    // Part::bytes: Data stays in memory
    // - Can be cleared after use: std::mem::drop(data)
    // - Sensitive data visible in memory dumps
    // - Data copied multiple times (Vec -> Part -> Network)
    
    // Part::file: Data on disk
    // - File system permissions apply
    // - Data read directly from disk
    // - Less memory exposure for sensitive data
    // - File can be securely deleted after upload
    
    // For sensitive data:
    let secret_data = load_encrypted();  // Decrypt to memory
    let part = multipart::Part::bytes(secret_data.clone());
    // upload...
    std::mem::drop(secret_data);  // Clear memory
    
    // Or keep encrypted on disk:
    let part = multipart::Part::file("encrypted.bin").await.unwrap();
}

Consider security implications for sensitive data.

Summary Table

use reqwest::multipart;
 
fn summary() {
    // | Aspect              | Part::bytes              | Part::file              |
    // |---------------------|--------------------------|-------------------------|
    // | Memory usage        | Full data size           | Buffer (~64 KB)         |
    // | Startup time        | Slower (read all)        | Faster (open only)      |
    // | Large file support   | Memory limited           | Unlimited               |
    // | Concurrent uploads   | Memory constrained       | Many concurrent         |
    // | Data source         | In-memory Vec<u8>        | File path               |
    // | Modification        | Easy (already in memory) | Requires re-read        |
    // | Error timing        | At creation              | During send             |
    // | Use case            | Small, processed data    | Large files             |
}

Choose based on data size and memory constraints.

Synthesis

Quick reference:

use reqwest::multipart;
 
// Small data or in-memory content
let part = multipart::Part::bytes(data)
    .file_name("data.bin")
    .mime_str("application/octet-stream")
    .unwrap();
 
// Large files or memory efficiency
let part = multipart::Part::file("large_file.bin")
    .await?
    .file_name("large_file.bin")
    .mime_str("application/octet-stream")
    .unwrap();
 
// Custom readers for streaming
let part = multipart::Part::reader(cursor)
    .file_name("stream.bin");

When to use each:

use reqwest::multipart;
 
// Use Part::bytes when:
// - Data is already in memory (< 100 MB recommended)
// - Content is generated or processed
// - Need to modify data before upload
// - Implementing tests or mocks
// - Data is small and fits memory
 
// Use Part::file when:
// - Files are large (> 100 MB)
// - Memory is constrained
// - No processing needed
// - Concurrent uploads required
// - Files already on disk
// - Zero-copy efficiency needed

Key insight: Part::bytes and Part::file represent two fundamental approaches to HTTP body construction: eager loading versus lazy streaming. Part::bytes eagerly loads all content into a Vec<u8>, giving you immediate access to the data but requiring memory proportional to payload size. Part::file defers reading until transmission, opening the file and streaming chunks on-demand—this uses constant memory regardless of file size. The choice directly impacts your application's memory footprint, especially under concurrent load: uploading 100 files of 100 MB each with Part::bytes requires 10 GB of memory, while Part::file uses only ~6 MB (100 × 64 KB buffers). Use Part::bytes when data is already in memory, small, or needs processing; use Part::file for any file that could strain memory or when building scalable systems that handle concurrent uploads. The streaming behavior of Part::file also integrates naturally with async I/O, preventing the runtime from being blocked by large file reads.