How does `serde::Deserializer::deserialize_any` enable format-agnostic parsing of self-describing data?

serde::Deserializer::deserialize_any allows a deserializer to drive the parsing process by inspecting the data format and calling the appropriate visitor method based on what type it finds, enabling self-describing formats like JSON to be parsed without knowing the expected type in advance. Unlike typed deserialize methods like deserialize_u32 or deserialize_string where the caller requests a specific type, deserialize_any asks the deserializer to examine the next value and tell the visitor what it is. This enables parsing formats where type information is embedded in the data itself, allowing generic deserializers that work across any self-describing format.

The Deserializer Trait and deserialize_any

use serde::de::{Deserializer, Visitor};
use std::fmt;
 
// The Deserializer trait has many methods for specific types:
// - deserialize_bool
// - deserialize_u8, deserialize_u16, ...
// - deserialize_string, deserialize_str
// - deserialize_seq, deserialize_map
// - deserialize_any
 
// Most methods: caller requests a specific type
// deserialize_any: deserializer tells caller what type it found
 
fn deserializer_trait() {
    // Conceptually, the Deserializer trait has:
    // fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Self::Error>
    // where V: Visitor<'de>
    
    // The deserializer examines the next value and calls the
    // appropriate visitor method based on what it finds
}

deserialize_any is unique: the deserializer decides the type, not the caller.

Self-Describing Formats

use serde_json::json;
use serde::Deserialize;
 
fn self_describing_example() {
    // JSON is self-describing because:
    // - Numbers have distinct syntax from strings
    // - Booleans have distinct values (true/false)
    // - Arrays have [...] syntax
    // - Objects have {...} syntax
    // - Null has distinct value
    
    let json_value = json
!({
        "name": "Alice",
        "age": 30,
        "active": true,
        "tags": ["admin", "user"],
        "settings": null
    });
    
    // Each value's type is clear from the JSON syntax:
    // "name": string (has quotes)
    // "age": number (no quotes)
    // "active": boolean (true/false literal)
    // "tags": array ([...] brackets)
    // "settings": null (null literal)
    
    // When deserializing, JSON can tell you what type each value is
    // This enables deserialize_any
}

Self-describing formats encode type information in the data itself.

Non-Self-Describing Formats

// Bincode is NOT self-describing:
// - It's a binary format that doesn't include type tags
// - The deserializer must know what type to expect
// - deserialize_any cannot work because type is unknown
 
// Example: bincode encodes u32 as 4 bytes, u64 as 8 bytes
// If you don't know which to expect, you can't decode
 
// For non-self-describing formats, deserialize_any typically
// returns an error or makes a best-guess (often unsupported)
 
fn non_self_describing() {
    // bincode::deserialize_any would fail or be meaningless
    // because the bytes don't indicate their type
    
    // These serialize differently:
    let u32_bytes = bincode::serialize(&42u32).unwrap();  // 4 bytes
    let u64_bytes = bincode::serialize(&42u64).unwrap();  // 8 bytes
    
    // If you see 4 bytes, you can't know if it's u32, u16 x 2, etc.
    // Type must be known from context/schema
}

Non-self-describing formats require knowing the expected type.

The Visitor Pattern

use serde::de::{Visitor, Error};
use std::fmt;
 
// Visitor is the callback interface for deserialize_any
// The deserializer calls the appropriate method based on what it finds
 
struct AnyVisitor;
 
impl<'de> Visitor<'de> for AnyVisitor {
    type Value = String;  // What we return
    
    // Called when deserializer finds a string
    fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("Found string: {}", v))
    }
    
    // Called when deserializer finds a number
    fn visit_u64<E>(self, v: u64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("Found number: {}", v))
    }
    
    // Called when deserializer finds a boolean
    fn visit_bool<E>(self, v: bool) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("Found boolean: {}", v))
    }
    
    // Called for arrays
    fn visit_seq<A>(self, seq: A) -> Result<Self::Value, A::Error>
    where A: serde::de::SeqAccess<'de>
    {
        Ok(format!("Found array"))
    }
    
    // ... and many more visit_* methods
}

The Visitor receives callbacks for each type the deserializer encounters.

deserialize_any in Action

use serde::de::{Deserializer, Visitor, Error};
use serde_json::Deserializer;
use std::fmt;
 
struct TypeDetectingVisitor;
 
impl<'de> Visitor<'de> for TypeDetectingVisitor {
    type Value = String;
    
    fn expecting(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
        write!(fmt, "any valid JSON value")
    }
    
    fn visit_bool<E>(self, v: bool) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("boolean: {}", v))
    }
    
    fn visit_u64<E>(self, v: u64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("unsigned integer: {}", v))
    }
    
    fn visit_i64<E>(self, v: i64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("signed integer: {}", v))
    }
    
    fn visit_f64<E>(self, v: f64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("float: {}", v))
    }
    
    fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("string: {}", v))
    }
    
    fn visit_none<E>(self) -> Result<Self::Value, E>
    where E: Error
    {
        Ok("null".to_string())
    }
    
    fn visit_unit<E>(self) -> Result<Self::Value, E>
    where E: Error
    {
        Ok("unit/null".to_string())
    }
    
    fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
    where A: serde::de::SeqAccess<'de>
    {
        let mut items = Vec::new();
        while let Some(item) = seq.next_element::<serde_json::Value>()? {
            items.push(item);
        }
        Ok(format!("array with {} items", items.len()))
    }
    
    fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>
    where A: serde::de::MapAccess<'de>
    {
        let mut count = 0;
        while map.next_entry::<String, serde_json::Value>()?.is_some() {
            count += 1;
        }
        Ok(format!("object with {} fields", count))
    }
}
 
fn detect_json_types() {
    // Each value's type is detected from JSON syntax
    let json_values = [
        ("true", "boolean: true"),
        ("false", "boolean: false"),
        ("42", "unsigned integer: 42"),
        ("-17", "signed integer: -17"),
        ("3.14", "float: 3.14"),
        ("\"hello\"", "string: hello"),
        ("null", "unit/null"),
        ("[1, 2, 3]", "array with 3 items"),
        ("{\"a\": 1}", "object with 1 fields"),
    ];
    
    for (json, expected) in json_values {
        let mut deserializer = Deserializer::from_str(json);
        let result = deserializer.deserialize_any(TypeDetectingVisitor).unwrap();
        assert_eq!(result, expected);
    }
}

deserialize_any with a visitor detects types from the JSON syntax.

Generic Parsing Without Knowing Type

use serde_json::Value;
 
fn parse_any_type() {
    // serde_json::Value uses deserialize_any internally
    // It can represent any JSON value
    
    let json = r#"{
        "name": "Alice",
        "age": 30,
        "tags": ["admin", "user"],
        "active": true
    }"#;
    
    let value: Value = serde_json::from_str(json).unwrap();
    
    // Value can be any JSON type
    if let Some(obj) = value.as_object() {
        // Each field value is also a Value
        let name = obj.get("name").unwrap();
        println!("Name is string: {}", name.is_string());
        
        let age = obj.get("age").unwrap();
        println!("Age is number: {}", age.is_u64());
        
        let tags = obj.get("tags").unwrap();
        println!("Tags is array: {}", tags.is_array());
    }
}

Value types use deserialize_any to accept any valid format value.

Implementing deserialize_any for a Format

use serde::de::{Deserializer, Visitor, Error};
use std::fmt;
 
// Simplified example: a deserializer for a self-describing format
struct MyFormatDeserializer<'a> {
    input: &'a str,
}
 
impl<'de, 'a> Deserializer<'de> for MyFormatDeserializer<'a> {
    type Error = serde::de::value::Error;
    
    fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Self::Error>
    where V: Visitor<'de>
    {
        // Examine the next token and call appropriate visitor method
        let input = self.input.trim();
        
        if input == "true" {
            visitor.visit_bool(true)
        } else if input == "false" {
            visitor.visit_bool(false)
        } else if input == "null" {
            visitor.visit_unit()
        } else if input.starts_with('"') {
            // Parse string
            visitor.visit_str(input.trim_matches('"'))
        } else if input.starts_with('[') {
            // Parse array
            visitor.visit_seq(/* ... */)
        } else if input.starts_with('{') {
            // Parse object
            visitor.visit_map(/* ... */)
        } else {
            // Try to parse as number
            if let Ok(n) = input.parse::<i64>() {
                visitor.visit_i64(n)
            } else if let Ok(n) = input.parse::<u64>() {
                visitor.visit_u64(n)
            } else if let Ok(n) = input.parse::<f64>() {
                visitor.visit_f64(n)
            } else {
                Err(Self::Error::custom("unknown type"))
            }
        }
    }
    
    // Other deserialize_* methods can delegate to deserialize_any
    // for self-describing formats
    
    fn deserialize_bool<V>(self, visitor: V) -> Result<V::Value, Self::Error>
    where V: Visitor<'de>
    {
        self.deserialize_any(visitor)
    }
    
    // ... implement other methods ...
}

Self-describing formats implement deserialize_any by parsing and calling the appropriate visitor method.

Why deserialize_any Matters for Format Agnostic Code

use serde::de::DeserializeOwned;
use serde_json;
// use bincode;
 
fn format_agnostic_parsing<T: DeserializeOwned>(data: &[u8]) -> Result<T, Box<dyn std::error::Error>> {
    // With self-describing formats, the same code works
    
    // JSON uses deserialize_any internally to parse unknown types
    let result: T = serde_json::from_slice(data)?;
    
    // MessagePack is also self-describing
    // let result: T = rmp_serde::from_slice(data)?;
    
    // Both use deserialize_any to determine type from data
    
    Ok(result)
}
 
fn why_it_matters() {
    // Consider a generic function that must work with multiple formats
    // The function doesn't know the type T in advance
    // The format tells us what type the value is
    
    // deserialize_any enables:
    // 1. serde_json::Value - can hold any JSON value
    // 2. Generic deserializers - parse without knowing type
    // 3. Format adapters - convert between formats
    
    // Without deserialize_any, we'd need to know the type
    // before we could parse, which defeats generic parsing
}

deserialize_any enables generic deserializers that work without knowing the expected type.

The Trade-off: deserialize_any vs Typed Methods

use serde::de::{Deserializer, Visitor, Error};
use serde_json::Deserializer;
 
fn typed_vs_any() {
    // Typed deserialization: caller requests specific type
    let mut de = Deserializer::from_str("42");
    let value: u32 = de.deserialize_u32(/* visitor */).unwrap();
    // Deserializer knows to parse as u32
    
    // deserialize_any: deserializer decides type
    let mut de = Deserializer::from_str("42");
    let value = de.deserialize_any(TypeDetectingVisitor).unwrap();
    // Deserializer examines "42", sees it's a number, calls visit_u64
    
    // For self-describing formats:
    // - deserialize_any works because type is in data
    // - deserialize_u32 also works, validates type matches
    
    // For non-self-describing formats:
    // - deserialize_any may fail (can't determine type)
    // - deserialize_u32 works (caller tells type)
}

Typed methods specify what to parse; deserialize_any discovers what's there.

Deserializing Dynamic Types

use serde_json::Value;
use std::collections::HashMap;
 
fn dynamic_deserialization() {
    // Common pattern: deserialize any JSON, then process based on type
    
    let json = r#"{
        "type": "user",
        "data": {
            "id": 123,
            "name": "Alice"
        }
    }"#;
    
    let value: Value = serde_json::from_str(json).unwrap();
    
    // Examine type field to determine how to process
    if let Some(type_str) = value.get("type").and_then(|v| v.as_str()) {
        match type_str {
            "user" => {
                let data = value.get("data").unwrap();
                let id = data.get("id").and_then(|v| v.as_u64()).unwrap();
                let name = data.get("name").and_then(|v| v.as_str()).unwrap();
                println!("User {} ({})", name, id);
            }
            "product" => {
                // Handle product type
            }
            _ => {}
        }
    }
    
    // This pattern is fundamental for:
    // - Polymorphic deserialization
    // - API responses with varying structures
    // - Configuration with dynamic types
}

Dynamic deserialization uses deserialize_any to handle unknown types.

Efficiency Considerations

use serde::Deserialize;
 
fn efficiency_comparison() {
    // Typed deserialization can be more efficient
    
    #[derive(Deserialize)]
    struct Point {
        x: f64,
        y: f64,
    }
    
    let json = r#"{"x": 1.5, "y": 2.5}"#;
    
    // Typed: deserializer knows to expect f64, f64
    // Can skip type detection overhead
    let point: Point = serde_json::from_str(json).unwrap();
    
    // deserialize_any: must examine each value's type
    // More flexible, but has detection overhead
    
    // For non-self-describing formats like bincode:
    // - Typed is required (no type info in data)
    // - deserialize_any would fail or guess wrong
    
    // For self-describing formats like JSON:
    // - Both work
    // - deserialize_any is needed for Value/dynamic
    // - Typed can be slightly faster for known structures
}

Typed deserialization can skip type detection overhead for known structures.

deserialize_any and Ignored Values

use serde::de::{Deserializer, Visitor, IgnoredAny};
use serde_json::Deserializer;
 
fn skip_values() {
    // IgnoredAny uses deserialize_any to skip any value
    // Useful for ignoring unknown fields
    
    let json = r#"{"important": 42, "unknown": [1, 2, 3], "extra": {"nested": true}}"#;
    
    // IgnoredAny can accept and discard any type
    let mut de = Deserializer::from_str(json);
    
    // Parse the important field
    // Skip unknown and extra fields
    // IgnoredAny uses deserialize_any to consume any value
    
    // This is how #[serde(ignore)] works internally
}

IgnoredAny uses deserialize_any to skip values of any type.

Format Detection and deserialize_any

use serde::Deserialize;
use std::collections::HashMap;
 
fn format_agnostic_processing() {
    // With deserialize_any, code can be format-agnostic
    
    // Process config from any self-describing format
    fn process_config(data: &[u8]) -> HashMap<String, serde_json::Value> {
        // Could be JSON, MessagePack, etc.
        serde_json::from_slice(data).unwrap()
    }
    
    // The same Value type works for any self-describing format
    // because all use deserialize_any internally
    
    // JSON Value
    let json_data = br#"{"key": "value"}"#;
    let json_value: serde_json::Value = serde_json::from_slice(json_data).unwrap();
    
    // MessagePack would work similarly
    // let msgpack_value: rmpv::Value = rmp_serde::from_slice(msgpack_data).unwrap();
    
    // Both use deserialize_any to build their Value types
    // Both can represent any valid value from their format
}

deserialize_any enables Value types that work across self-describing formats.

Handling Unknown Types in deserialize_any

use serde::de::{Visitor, Error};
use std::fmt;
 
struct FlexibleVisitor;
 
impl<'de> Visitor<'de> for FlexibleVisitor {
    type Value = String;
    
    fn expecting(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
        write!(fmt, "any value")
    }
    
    // Handle all possible types
    
    fn visit_bool<E>(self, v: bool) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("bool({})", v))
    }
    
    fn visit_i64<E>(self, v: i64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("int({})", v))
    }
    
    fn visit_u64<E>(self, v: u64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("uint({})", v))
    }
    
    fn visit_f64<E>(self, v: f64) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("float({})", v))
    }
    
    fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
    where E: Error
    {
        Ok(format!("string(\"{}\")", v))
    }
    
    fn visit_none<E>(self) -> Result<Self::Value, E>
    where E: Error
    {
        Ok("null".to_string())
    }
    
    fn visit_unit<E>(self) -> Result<Self::Value, E>
    where E: Error
    {
        Ok("unit".to_string())
    }
    
    fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
    where A: serde::de::SeqAccess<'de>
    {
        let mut items = Vec::new();
        while let Some(item) = seq.next_element::<serde_json::Value>()? {
            items.push(item);
        }
        Ok(format!("array({:?})", items))
    }
    
    fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>
    where A: serde::de::MapAccess<'de>
    {
        let mut entries = Vec::new();
        while let Some((key, value)) = map.next_entry::<String, serde_json::Value>()? {
            entries.push((key, value));
        }
        Ok(format!("object({:?})", entries))
    }
}

A complete visitor handles all possible types that deserialize_any might report.

derive(Deserialize) and deserialize_any

use serde::Deserialize;
 
#[derive(Deserialize)]
struct User {
    name: String,
    age: u32,
}
 
fn derive_deserialize() {
    let json = r#"{"name": "Alice", "age": 30}"#;
    
    // When deriving Deserialize:
    // - Field types are known (name: String, age: u32)
    // - Uses typed deserialize methods for each field
    // - deserialize_string for "name"
    // - deserialize_u32 for "age"
    
    // The derived code looks something like:
    // fn deserialize<D>(deserializer: D) -> Result<User, D::Error>
    // where D: Deserializer
    // {
    //     // Uses deserialize_map, then deserialize_string, deserialize_u32
    // }
    
    // It does NOT use deserialize_any for fields with known types
    
    let user: User = serde_json::from_str(json).unwrap();
    assert_eq!(user.name, "Alice");
    assert_eq!(user.age, 30);
}

Derived deserializers use typed methods for known field types.

When deserialize_any is Essential

use serde::Deserialize;
use serde_json::Value;
 
fn essential_use_cases() {
    // 1. serde_json::Value and similar types
    let value: Value = serde_json::from_str(r#"{"any": [1, 2, "three"]}"#).unwrap();
    // Must accept any JSON value, uses deserialize_any
    
    // 2. Deserializing trait objects / polymorphic types
    // Use type field to determine concrete type
    #[derive(Deserialize)]
    #[serde(tag = "type")]
    enum Message {
        #[serde(rename = "ping")]
        Ping { id: u64 },
        #[serde(rename = "pong")]
        Pong { id: u64, result: String },
    }
    // Internally uses deserialize_any to read type tag first
    
    // 3. Format conversion
    // Convert between self-describing formats
    // Parse to Value (using deserialize_any), serialize to other format
    
    // 4. Unknown structure handling
    // APIs that return varying structures
    // Config files with dynamic content
}

deserialize_any is essential for Value types and dynamic structures.

Synthesis

How deserialize_any enables format-agnostic parsing:

// Self-describing format: type information is IN the data
// - JSON: "hello" is string, 42 is number, true is bool
// - MessagePack: type tags before values
 
// deserialize_any: deserializer examines data, calls visitor
// - Sees "hello" -> calls visitor.visit_str("hello")
// - Sees 42 -> calls visitor.visit_u64(42)
// - Sees true -> calls visitor.visit_bool(true)
 
// This enables:
// 1. Value types that hold any value (serde_json::Value)
// 2. Generic parsing without knowing expected type
// 3. Format-agnostic deserializers
// 4. Polymorphic deserialization

The key insight:

// For typed deserialize methods:
// deserialize_u32(visitor) -> "Parse this as u32"
// deserialize_string(visitor) -> "Parse this as string"
 
// For deserialize_any:
// deserialize_any(visitor) -> "Parse whatever is there, tell visitor the type"
 
// Self-describing formats CAN implement deserialize_any
// because they know the type from the data
 
// Non-self-describing formats CANNOT implement deserialize_any
// because type information is not in the data

When to use deserialize_any:

// Use deserialize_any when:
// 1. Implementing a deserializer for a self-describing format
// 2. Creating Value types (serde_json::Value, etc.)
// 3. Handling unknown or dynamic structures
// 4. Building format-agnostic parsers
 
// Use typed deserialize methods when:
// 1. You know the expected type (derive Deserialize)
// 2. Efficiency matters (skip type detection)
// 3. Non-self-describing formats (bincode)

The visitor pattern makes it work:

// deserialize_any is a callback-based API:
// 1. Caller provides visitor with callbacks for all types
// 2. Deserializer examines data
// 3. Deserializer calls appropriate visitor method
// 4. Visitor returns parsed value
 
// This inversion of control is key:
// - Caller doesn't need to know type in advance
// - Deserializer tells caller what type it found
// - Format-agnostic code becomes possible

The fundamental insight is that deserialize_any inverts the normal deserialization control flow: instead of the caller requesting a specific type and the deserializer trying to produce it, the deserializer examines the data and tells the visitor what type it found. This is only possible with self-describing formats where type information is embedded in the data itself. JSON values carry their type (strings have quotes, numbers don't, arrays have brackets), so a JSON deserializer can call visit_str when it sees quotes, visit_u64 when it sees a number literal, etc. This enables serde_json::Value and similar types that can hold any valid value from the format, making format-agnostic parsing possible.

Loading page…

How does serde::Deserializer::deserialize_any enable format-agnostic parsing of self-describing data?

The Deserializer Trait and deserialize_any

Self-Describing Formats

Non-Self-Describing Formats

The Visitor Pattern

deserialize_any in Action

Generic Parsing Without Knowing Type

Implementing deserialize_any for a Format

Why deserialize_any Matters for Format Agnostic Code

The Trade-off: deserialize_any vs Typed Methods

Deserializing Dynamic Types

Efficiency Considerations

deserialize_any and Ignored Values

Format Detection and deserialize_any

Handling Unknown Types in deserialize_any

derive(Deserialize) and deserialize_any

When deserialize_any is Essential

Synthesis

How does serde::Deserializer::deserialize_any enable format-agnostic parsing of self-describing data?

The Deserializer Trait and deserialize_any

Self-Describing Formats

Non-Self-Describing Formats

The Visitor Pattern

deserialize_any in Action

Generic Parsing Without Knowing Type

Implementing deserialize_any for a Format

Why deserialize_any Matters for Format Agnostic Code

The Trade-off: deserialize_any vs Typed Methods

Deserializing Dynamic Types

Efficiency Considerations

deserialize_any and Ignored Values

Format Detection and deserialize_any

Handling Unknown Types in deserialize_any

derive(Deserialize) and deserialize_any

When deserialize_any is Essential

Synthesis

How does `serde::Deserializer::deserialize_any` enable format-agnostic parsing of self-describing data?

How does `serde::Deserializer::deserialize_any` enable format-agnostic parsing of self-describing data?