Loading pageâŚ
Rust walkthroughs
Loading pageâŚ
axum::Json deserialize request bodies and what are the implications for large payloads?axum::Json is an extractor that fully buffers the request body into memory before deserializing it with serde_json, which has significant implications for large payloads. The entire body must fit in memory, and deserialization happens after the complete body is received. For large JSON payloads, this can lead to high memory usage, slower response times while buffering, and potential denial-of-service vulnerabilities. Axum provides DefaultBodyLimit to restrict request sizes, and alternatives like streaming parsers or incremental deserialization can handle large payloads more efficiently.
use axum::{
extract::Json,
routing::post,
Router,
};
use serde::Deserialize;
#[derive(Deserialize)]
struct User {
name: String,
email: String,
}
async fn create_user(Json(user): Json<User>) -> String {
format!("Created user: {}", user.name)
}
#[tokio::main]
async fn main() {
let app = Router::new()
.route("/users", post(create_user));
let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
axum::serve(listener, app).await.unwrap();
}Json extracts and deserializes the request body into a struct.
use axum::extract::Json;
use serde::Deserialize;
#[derive(Deserialize, Debug)]
struct Payload {
id: u32,
items: Vec<String>,
}
async fn handler(Json(payload): Json<Payload>) -> &'static str {
// At this point:
// 1. Entire request body has been buffered
// 2. Body has been parsed as JSON
// 3. Deserialized into Payload struct
// 4. Original bytes are deallocated
println!("Received: {:?}", payload);
"ok"
}Json buffers the full body, then deserializes.
use axum::{
extract::Json,
routing::post,
Router,
};
use serde::Deserialize;
#[derive(Deserialize)]
struct LargePayload {
data: Vec<Vec<String>>, // Can grow arbitrarily large
}
async fn process(Json(payload): Json<LargePayload>) -> &'static str {
// Memory usage:
// 1. Raw bytes of entire request body
// 2. Deserialized struct with all allocations
// 3. Both exist momentarily during deserialization
println!("Items: {}", payload.data.len());
"processed"
}Large payloads temporarily hold both raw bytes and deserialized data.
use axum::{
extract::DefaultBodyLimit,
routing::post,
Router,
};
#[tokio::main]
async fn main() {
// Default limit is 2MB for regular routes
let app = Router::new()
.route("/upload", post(upload))
// Default limit applies automatically
.route("/large", post(large_upload));
// Requests over 2MB return 413 Payload Too Large
}
async fn upload() -> &'static str {
"ok"
}
async fn large_upload() -> &'static str {
"ok"
}Axum defaults to 2MB body limit for protection.
use axum::{
extract::DefaultBodyLimit,
routing::post,
Router,
};
#[tokio::main]
async fn main() {
let app = Router::new()
// Global limit: 1MB
.layer(DefaultBodyLimit::max(1024 * 1024))
.route("/small", post(small_handler))
.route("/medium", post(medium_handler));
// All routes limited to 1MB
}
async fn small_handler() -> &'static str {
"ok"
}
async fn medium_handler() -> &'static str {
"ok"
}Set global limits with DefaultBodyLimit middleware.
use axum::{
extract::{DefaultBodyLimit, Json},
routing::post,
Router,
};
use serde::Deserialize;
#[derive(Deserialize)]
struct Upload {
data: String,
}
#[tokio::main]
async fn main() {
let app = Router::new()
// This route: 1KB limit
.route("/tiny", post(tiny_handler))
// This route: 10MB limit
.route("/upload", post(upload_handler))
// Global default for other routes
.route("/default", post(default_handler));
let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
axum::serve(listener, app).await.unwrap();
}
async fn tiny_handler() -> &'static str {
"tiny"
}
async fn upload_handler() -> &'static str {
"upload"
}
async fn default_handler() -> &'static str {
"default"
}Configure limits per route based on expected payload sizes.
use axum::{
body::Body,
extract::BodyStream,
http::StatusCode,
response::IntoResponse,
};
use futures::StreamExt;
async fn stream_upload(mut body: BodyStream) -> Result<String, StatusCode> {
let mut total_size = 0u64;
// Process body as stream of chunks
while let Some(chunk) = body.next().await {
let chunk = chunk.map_err(|_| StatusCode::BAD_REQUEST)?;
total_size += chunk.len() as u64;
// Process chunk incrementally
// No full buffering required
}
Ok(format!("Received {} bytes", total_size))
}BodyStream processes chunks without full buffering.
use axum::{
body::Body,
http::StatusCode,
Json,
};
use serde::Deserialize;
// For truly large JSON, consider streaming parsers
// like serde_json::StreamDeserializer
#[derive(Deserialize)]
struct Record {
id: u32,
value: String,
}
// Standard Json extractor loads entire body first
async fn standard_handler(Json(records): Json<Vec<Record>>) -> String {
format!("Received {} records", records.len())
}
// For large arrays, consider custom streaming
async fn streaming_handler(body: Body) -> Result<String, StatusCode> {
// Custom streaming implementation needed
// using serde_json::from_reader or similar
unimplemented!("Use streaming JSON parser for large payloads")
}For large JSON arrays, stream records incrementally.
use axum::{
extract::Json,
http::StatusCode,
response::IntoResponse,
};
use serde::Deserialize;
#[derive(Deserialize)]
struct User {
name: String,
age: u32,
}
async fn create_user(body: Result<Json<User>, axum::extract::rejection::JsonRejection>) -> impl IntoResponse {
match body {
Ok(Json(user)) => (StatusCode::OK, format!("Created: {}", user.name)),
Err(rejection) => {
// JsonRejection contains error details
let body_text = rejection.body_text();
(rejection.status(), body_text)
}
}
}Handle deserialization errors with Result<Json<T>, JsonRejection>.
use axum::extract::rejection::JsonRejection;
fn handle_rejection(rejection: JsonRejection) -> (StatusCode, String) {
match rejection {
JsonRejection::JsonDataError(e) => {
(StatusCode::BAD_REQUEST, format!("Invalid JSON: {}", e))
}
JsonRejection::JsonSyntaxError(e) => {
(StatusCode::BAD_REQUEST, format!("Syntax error: {}", e))
}
JsonRejection::MissingJsonContentType(_) => {
(StatusCode::BAD_REQUEST, "Missing Content-Type: application/json".to_string())
}
JsonRejection::BytesRejection(e) => {
(StatusCode::INTERNAL_SERVER_ERROR, format!("Body error: {}", e))
}
_ => (StatusCode::BAD_REQUEST, "Invalid request".to_string()),
}
}JsonRejection variants provide specific error information.
use axum::{
extract::Json,
http::StatusCode,
};
use serde::Deserialize;
#[derive(Deserialize)]
struct Data {
value: String,
}
async fn handler(Json(data): Json<Data>) -> Result<String, StatusCode> {
// Json extractor validates Content-Type: application/json
// Returns MissingJsonContentType error if wrong/missing
Ok(format!("Received: {}", data.value))
}
// Request must include:
// Content-Type: application/json
// Body: valid JSON matching structJson requires Content-Type: application/json header.
use axum::{
body::Bytes,
extract::Json,
};
use serde::Deserialize;
#[derive(Deserialize)]
struct Payload {
id: u32,
}
// Json: validates content-type, parses JSON, deserializes
async fn json_handler(Json(payload): Json<Payload>) -> String {
format!("ID: {}", payload.id)
}
// Bytes: raw body, no parsing
async fn bytes_handler(body: Bytes) -> Result<String, StatusCode> {
// Manual parsing needed
let payload: Payload = serde_json::from_slice(&body)
.map_err(|_| StatusCode::BAD_REQUEST)?;
Ok(format!("ID: {}", payload.id))
}Json handles content-type validation and parsing automatically.
use axum::Json;
use serde::Deserialize;
#[derive(Deserialize)]
struct Nested {
level1: Level1,
}
struct Level1 {
level2: Level2,
}
struct Level2 {
data: String,
}
async fn process(Json(payload): Json<Nested>) -> String {
// Memory lifecycle:
// 1. Request body arrives -> Bytes allocated
// 2. Bytes parsed as JSON -> Intermediate structures
// 3. Deserialized into structs -> Final allocations
// 4. Bytes deallocated (only struct remains)
// Peak memory = request size + deserialized size + parsing overhead
"processed".to_string()
}Peak memory includes both raw bytes and parsed structures.
use axum::{
extract::DefaultBodyLimit,
routing::post,
Router,
};
#[tokio::main]
async fn main() {
// Prevent DoS by limiting body size
let app = Router::new()
.route("/api/upload", post(upload))
// Limit to 10MB - adjust based on your needs
.layer(DefaultBodyLimit::max(10 * 1024 * 1024));
let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
axum::serve(listener, app).await.unwrap();
}
async fn upload() -> &'static str {
"uploaded"
}Always set body limits in production to prevent DoS attacks.
use axum::{
extract::Multipart,
http::StatusCode,
};
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
async fn upload_file(mut multipart: Multipart) -> Result<String, StatusCode> {
while let Some(field) = multipart.next_field().await.map_err(|_| StatusCode::BAD_REQUEST)? {
let filename = field.file_name().unwrap_or("unknown").to_string();
// Stream file to disk instead of memory
let mut file = File::create(format!("/tmp/{}", filename))
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
while let Some(chunk) = field.chunk().await.map_err(|_| StatusCode::BAD_REQUEST)? {
file.write_all(&chunk)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
}
}
Ok("uploaded".to_string())
}For large files, use Multipart to stream to disk.
use axum::{
extract::BodyStream,
http::StatusCode,
};
use futures::StreamExt;
use serde_json::Deserializer;
use serde::Deserialize;
use std::io::Cursor;
#[derive(Deserialize)]
struct Item {
id: u32,
}
async fn process_items(body: BodyStream) -> Result<String, StatusCode> {
let mut buffer = Vec::new();
let mut count = 0;
// Collect chunks with limit
for await chunk in body {
let chunk = chunk.map_err(|_| StatusCode::BAD_REQUEST)?;
// Implement size limit manually
if buffer.len() + chunk.len() > 1024 * 1024 {
return Err(StatusCode::PAYLOAD_TOO_LARGE);
}
buffer.extend_from_slice(&chunk);
}
// Parse as array of items
let items: Vec<Item> = serde_json::from_slice(&buffer)
.map_err(|_| StatusCode::BAD_REQUEST)?;
Ok(format!("Processed {} items", items.len()))
}Implement custom streaming for specific use cases.
use axum::{
extract::Json,
routing::post,
Router,
};
use std::time::Duration;
use tokio::time::timeout;
use serde::Deserialize;
#[derive(Deserialize)]
struct LargePayload {
items: Vec<String>,
}
async fn handler_with_timeout(Json(payload): Json<LargePayload>) -> Result<String, &'static str> {
// Add timeout for processing large payloads
let result = timeout(
Duration::from_secs(30),
process_payload(payload)
).await.map_err(|_| "timeout")?;
Ok(result)
}
async fn process_payload(payload: LargePayload) -> String {
format!("Items: {}", payload.items.len())
}Add timeouts to prevent slow requests from blocking resources.
| Approach | Memory Use | Complexity | Use Case |
|----------|------------|------------|----------|
| Json<T> | Full body + struct | Low | Small payloads |
| BodyStream | Chunk at a time | Medium | Large/raw data |
| Multipart | Stream to disk | Medium | File uploads |
| Bytes | Full body | Low | Need raw bytes |
| Custom streaming | Configurable | High | Very large JSON |
axum::Json deserializes by fully buffering the request body into memory before parsing, which has critical implications:
Memory allocation: The entire request body is buffered as bytes, then deserialized into the target struct. Peak memory includes both allocations plus parsing overhead. For a 10MB JSON payload, you might temporarily need 20-30MB of memory.
Denial of service risk: Without body limits, attackers can send arbitrarily large payloads to exhaust server memory. Always configure DefaultBodyLimit in productionâAxum's default 2MB limit is a safety net, not a feature to rely on.
Deserialization timing: Parsing only begins after the entire body is received. This delays response time and holds resources longer than streaming alternatives.
Alternatives for large payloads:
BodyStream processes chunks incrementally without full bufferingMultipart streams file uploads directly to diskKey insight: Json is optimized for developer ergonomics and typical API payloads, not large file processing. For payloads exceeding a few megabytes, switch to streaming approaches. The convenience of automatic deserialization comes with the cost of full-body buffering. Configure body limits appropriate to your application, and use streaming extractors when payload sizes are unpredictable or large.