How does serde::de::Unexpected::Unit interact with #[serde(deny_unknown_fields)] for strict deserialization validation?
Unexpected::Unit is an enum variant that represents the type mismatch between what the input data contained (a unit value null or ()) and what the deserializer expected, while deny_unknown_fields is an attribute that rejects any unrecognized fields during deserialization—these are orthogonal mechanisms that operate at different stages of deserialization. When a deserializer encounters a unit value where a different type is expected, it produces an Unexpected::Unit error; when deny_unknown_fields encounters an unknown field, it produces a different error. These mechanisms can combine in scenarios where a struct with deny_unknown_fields receives unexpected field values, but they operate independently in the deserialization pipeline.
Understanding Unexpected::Unit
use serde::de::{Unexpected, Error};
// Unexpected is an enum representing what was found in the input
// when something else was expected:
pub enum Unexpected<'a> {
Bool(bool),
Unsigned(u64),
Signed(i64),
Float(f64),
Char(char),
Str(&'a str),
String(&'a str),
Bytes(&'a [u8]),
// Unit represents null, (), or empty value
Unit,
Option,
NewtypeStruct,
Seq,
Map,
Enum(&'a str),
// ... other variants
}
// Unit specifically represents:
// - JSON null
// - YAML null
// - Rust () unit type
// - Empty TOML value
// Example: deserializing into a type that doesn't accept unit:
use serde::Deserialize;
fn demonstrate_unexpected_unit() {
let json = r#"null"#;
// Attempting to deserialize null into a type expecting a value
let result: Result<i32, _> = serde_json::from_str(json);
match result {
Err(e) => {
// The error will contain Unexpected::Unit
// because the input was null (unit) but i32 was expected
println!("Error: {}", e);
// Output: "invalid type: null, expected i32"
}
Ok(_) => unreachable!(),
}
// Another example with struct fields:
#[derive(Debug, Deserialize)]
struct Person {
name: String,
age: u32,
}
let json = r#"{"name": null, "age": 30}"#;
let result: Result<Person, _> = serde_json::from_str(json);
// Error: invalid type: null, expected a string
// The deserializer found Unit (null) where String was expected
}Unexpected::Unit is produced by deserializers when they encounter a null/unit value where a different type is expected.
The deny_unknown_fields Attribute
use serde::Deserialize;
// deny_unknown_fields rejects any fields not defined in the struct:
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct Config {
name: String,
value: i32,
}
fn demonstrate_deny_unknown() {
// Valid: only known fields
let valid = r#"{"name": "test", "value": 42}"#;
let config: Config = serde_json::from_str(valid).unwrap();
println!("{:?}", config); // Config { name: "test", value: 42 }
// Invalid: unknown field present
let invalid = r#"{"name": "test", "value": 42, "extra": "field"}"#;
let result: Result<Config, _> = serde_json::from_str(invalid);
match result {
Err(e) => {
// Error about unknown field, NOT about Unexpected::Unit
println!("Error: {}", e);
// Output: "unknown field `extra`, expected `name` or `value`"
}
Ok(_) => unreachable!(),
}
// Without deny_unknown_fields, extra fields are silently ignored:
#[derive(Debug, Deserialize)]
struct LaxConfig {
name: String,
value: i32,
}
let with_extra = r#"{"name": "test", "value": 42, "extra": "ignored"}"#;
let lax: LaxConfig = serde_json::from_str(with_extra).unwrap();
println!("{:?}", lax); // LaxConfig { name: "test", value: 42 }
// "extra" field silently ignored
}deny_unknown_fields produces errors for unknown field names, regardless of the field values' types.
How They Operate at Different Stages
use serde::Deserialize;
// These mechanisms operate at different points in deserialization:
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct User {
id: u64,
name: String,
active: bool,
}
fn demonstrate_stages() {
// Stage 1: Field name recognition
// deny_unknown_fields operates here
let unknown_field = r#"{"id": 1, "name": "Alice", "active": true, "role": "admin"}"#;
let result: Result<User, _> = serde_json::from_str(unknown_field);
match result {
Err(e) => {
// Error at field name stage: "role" is not recognized
// The value "admin" was never examined
println!("Field name error: {}", e);
}
_ => unreachable!(),
}
// Stage 2: Value type validation
// Unexpected::Unit operates here
let null_field = r#"{"id": null, "name": "Alice", "active": true}"#;
let result: Result<User, _> = serde_json::from_str(null_field);
match result {
Err(e) => {
// Error at value stage: null is not u64
// The field name "id" was recognized
// The value null triggered Unexpected::Unit
println!("Value type error: {}", e);
// Output: "invalid type: null, expected u64"
}
_ => unreachable!(),
}
// Combined: both mechanisms can trigger in sequence
let both_errors = r#"{"id": null, "name": "Alice", "extra": "field"}"#;
let result: Result<User, _> = serde_json::from_str(both_errors);
// The deserializer processes fields in order:
// 1. "id": recognized, null value triggers Unexpected::Unit
// 2. Processing stops at first error
// The "extra" field error is never reached
}deny_unknown_fields operates on field names before examining values; Unexpected::Unit operates on field values during type deserialization.
Error Types and Messages
use serde::de::{Unexpected, Error};
use serde_json::Error as JsonError;
// Different error types from different stages:
fn analyze_errors() {
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct Data {
count: i32,
label: String,
}
// Type mismatch error (Unexpected::Unit)
let null_input = r#"{"count": null, "label": "test"}"#;
let err = serde_json::from_str::<Data>(null_input).unwrap_err();
// The error contains:
// - Category: TypeMismatch (data was wrong type)
// - Unexpected::Unit (what was found)
// - Expected type (i32)
println!("Type error: {}", err);
// "invalid type: null, expected i32"
// Unknown field error (deny_unknown_fields)
let extra_field = r#"{"count": 5, "label": "test", "extra": null}"#;
let err = serde_json::from_str::<Data>(extra_field).unwrap_err();
println!("Unknown field error: {}", err);
// "unknown field `extra`, expected `count` or `label`"
// The errors are distinct:
// - Type errors mention "invalid type" and what was expected
// - Unknown field errors mention "unknown field" and available fields
}Error messages clearly distinguish between type mismatches (Unexpected::Unit) and unknown field errors.
When Both Appear Together
use serde::Deserialize;
// Scenarios where both mechanisms interact:
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct Complex {
required: String,
optional: Option<i32>,
}
fn combined_scenarios() {
// Case 1: Unknown field with null value
// The unknown field error takes precedence
let unknown_with_null = r#"{"required": "test", "extra": null}"#;
let result: Result<Complex, _> = serde_json::from_str(unknown_with_null);
// Error: unknown field `extra`, expected `required` or `optional`
// Even though "extra" has null value, the field name error triggers first
// Case 2: Known field with null where value expected
// The type mismatch (Unexpected::Unit) triggers
let null_required = r#"{"required": null}"#;
let result: Result<Complex, _> = serde_json::from_str(null_required);
// Error: invalid type: null, expected a string
// The field name was recognized, but the value was wrong type
// Case 3: Optional field accepts null (no Unexpected::Unit)
let null_optional = r#"{"required": "test", "optional": null}"#;
let data: Complex = serde_json::from_str(null_optional).unwrap();
// Success! Option<T> accepts null as None
// No Unexpected::Unit error because null is valid for Option
// Case 4: Missing required field
// Different error entirely
let missing_required = r#"{"optional": 5}"#;
let result: Result<Complex, _> = serde_json::from_str(missing_required);
// Error: missing field `required`
// Neither Unexpected::Unit nor unknown field error
}The order of field processing determines which error appears first—unknown fields are detected before field values are examined.
Deserializer Implementation Perspective
use serde::de::{self, Deserialize, DeserializeSeed, Error, Unexpected, Visitor};
// How a deserializer produces Unexpected::Unit:
struct SimpleDeserializer<'a> {
input: &'a str,
}
impl<'de, 'a> de::Deserializer<'de> for SimpleDeserializer<'a> {
type Error = de::value::Error;
fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
// Parse the input and dispatch to appropriate method
match self.input {
"null" => visitor.visit_unit(), // Produces unit value
_ => Err(de::Error::custom("unsupported input")),
}
}
fn deserialize_u32<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
// If input is "null", we expect a number but got unit
match self.input {
"null" => {
// This is where Unexpected::Unit appears
// The deserializer expected a number but found null
Err(de::Error::invalid_type(Unexpected::Unit, &"u32"))
}
s => {
// Try to parse as number
let n: u32 = s.parse().map_err(|_| de::Error::custom("not a number"))?;
visitor.visit_u32(n)
}
}
}
// Other methods...
}
// The Visitor receives values, but if type doesn't match,
// the deserializer returns an error with Unexpected::Unit
// Example usage:
fn test_deserializer() {
use serde::de::value::Error;
// Attempting to deserialize "null" as u32
let result: Result<u32, Error> = SimpleDeserializer { input: "null" }.deserialize_u32(de::value::SeqAccess::new(vec![].into_iter()));
// This would produce an error with Unexpected::Unit
// because the deserializer expected u32 but found null (unit)
}Deserializers produce Unexpected::Unit errors when the input contains null/unit where a non-unit type is expected.
Visitor Error Messages
use serde::de::{self, Visitor, Unexpected, Error};
// The Error trait provides methods for Unexpected variants:
struct MyError(String);
impl std::fmt::Display for MyError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}
impl std::error::Error for MyError {}
impl de::Error for MyError {
fn custom<T: std::fmt::Display>(msg: T) -> Self {
MyError(msg.to_string())
}
// The Error trait has methods like:
// - invalid_type(unexpected, expected)
// - invalid_value(unexpected, expected)
// - invalid_length(len, expected)
// - unknown_variant(variant, expected)
// - unknown_field(field, expected)
// - missing_field(field)
// - duplicate_field(field)
}
// Example: creating an Unexpected::Unit error manually:
fn create_unexpected_unit_error() -> MyError {
// When a visitor expects a specific type but receives unit:
de::Error::invalid_type(Unexpected::Unit, &"a non-null value")
}
// Example: creating an unknown field error:
fn create_unknown_field_error() -> MyError {
// deny_unknown_fields uses this internally:
de::Error::unknown_field("extra", &["name", "value"])
}
// Example: creating a missing field error:
fn create_missing_field_error() -> MyError {
de::Error::missing_field("required")
}
// The error messages:
fn print_error_examples() {
// Unexpected::Unit error:
// "invalid type: null, expected i32"
// Unknown field error:
// "unknown field `extra`, expected `name` or `value`"
// Missing field error:
// "missing field `required`"
}Unexpected::Unit is one of several Unexpected variants used to construct descriptive type mismatch errors.
Interaction with Option Types
use serde::Deserialize;
// Option<T> deserializes null as None, avoiding Unexpected::Unit:
#[derive(Debug, Deserialize)]
struct WithOptions {
name: String,
age: Option<u32>, // Accepts null or number
email: Option<String>,
}
fn option_handling() {
// Null is valid for Option fields:
let valid_null = r#"{"name": "Alice", "age": null}"#;
let data: WithOptions = serde_json::from_str(valid_null).unwrap();
assert_eq!(data.age, None);
// Number is also valid for Option:
let valid_number = r#"{"name": "Alice", "age": 30}"#;
let data: WithOptions = serde_json::from_str(valid_number).unwrap();
assert_eq!(data.age, Some(30));
// But null on non-Option field triggers Unexpected::Unit:
#[derive(Debug, Deserialize)]
struct WithoutOptions {
name: String,
age: u32, // Cannot be null
}
let invalid_null = r#"{"name": "Alice", "age": null}"#;
let result: Result<WithoutOptions, _> = serde_json::from_str(invalid_null);
assert!(result.is_err());
// Error: invalid type: null, expected u32
// With deny_unknown_fields, null in unknown field doesn't
// trigger Unexpected::Unit because the field error comes first:
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct StrictWithOptions {
name: String,
age: Option<u32>,
}
// Unknown field with null:
let unknown_null = r#"{"name": "Alice", "age": 30, "extra": null}"#;
let result: Result<StrictWithOptions, _> = serde_json::from_str(unknown_null);
// Error: unknown field `extra`
// Unexpected::Unit for "extra" field is never examined
// because field name check comes first
}Option<T> naturally accepts null values, preventing Unexpected::Unit errors for optional fields.
Custom Deserialize Implementations
use serde::{Deserialize, Deserializer, de::{self, Unexpected, Visitor}};
// Custom deserialize can choose how to handle Unexpected::Unit:
#[derive(Debug)]
struct StrictString(String);
impl<'de> Deserialize<'de> for StrictString {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
// Custom visitor that rejects null explicitly
struct StrictStringVisitor;
impl<'de> Visitor<'de> for StrictStringVisitor {
type Value = StrictString;
fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "a non-empty string, never null")
}
fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
where
E: de::Error,
{
Ok(StrictString(v.to_string()))
}
fn visit_string<E>(self, v: String) -> Result<Self::Value, E>
where
E: de::Error,
{
Ok(StrictString(v))
}
fn visit_unit<E>(self) -> Result<Self::Value, E>
where
E: de::Error,
{
// Explicitly reject null/unit with custom message
Err(de::Error::invalid_type(Unexpected::Unit, &"a string"))
}
fn visit_none<E>(self) -> Result<Self::Value, E>
where
E: de::Error,
{
// Also reject explicit None (for Option inner handling)
Err(de::Error::invalid_type(Unexpected::Unit, &"a string"))
}
}
deserializer.deserialize_string(StrictStringVisitor)
}
}
// Using the custom type:
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct Document {
title: StrictString,
content: StrictString,
}
fn test_custom_deserialize() {
// Valid input:
let valid = r#"{"title": "Hello", "content": "World"}"#;
let doc: Document = serde_json::from_str(valid).unwrap();
// Null title triggers Unexpected::Unit via custom visitor:
let null_title = r#"{"title": null, "content": "World"}"#;
let result: Result<Document, _> = serde_json::from_str(null_title);
assert!(result.is_err());
// Error: invalid type: null, expected a string
// Unknown field with null - deny_unknown_fields error first:
let unknown_field = r#"{"title": "Hello", "content": "World", "extra": null}"#;
let result: Result<Document, _> = serde_json::from_str(unknown_field);
assert!(result.is_err());
// Error: unknown field `extra`, expected `title` or `content`
}Custom Deserialize implementations control how Unexpected::Unit errors are produced through the Visitor trait.
Interaction with Default Values
use serde::Deserialize;
// default values can prevent Unexpected::Unit errors:
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct Config {
#[serde(default)]
name: String,
#[serde(default = "default_count")]
count: u32,
#[serde(default)]
enabled: bool,
}
fn default_count() -> u32 { 100 }
fn default_handling() {
// Missing fields use defaults instead of error:
let missing = r#"{}"#;
let config: Config = serde_json::from_str(missing).unwrap();
assert_eq!(config.name, "");
assert_eq!(config.count, 100);
assert_eq!(config.enabled, false);
// But null still triggers Unexpected::Unit:
let null_name = r#"{"name": null}"#;
let result: Result<Config, _> = serde_json::from_str(null_name);
assert!(result.is_err());
// Error: invalid type: null, expected a string
// default only handles MISSING fields, not NULL fields
// To accept null as default, use deserialize_with or Option:
#[derive(Debug, Deserialize)]
struct FlexConfig {
#[serde(default, deserialize_with = "deserialize_null_as_default")]
name: String,
}
fn deserialize_null_as_default<'de, D>(deserializer: D) -> Result<String, D::Error>
where
D: Deserializer<'de>,
{
use serde::de::Error;
let opt = Option::deserialize(deserializer)?;
Ok(opt.unwrap_or_default())
}
// Or simpler: use Option with default:
#[derive(Debug, Deserialize)]
struct SimplerConfig {
#[serde(default)]
name: Option<String>,
}
let null_accepted = r#"{"name": null}"#;
let config: SimplerConfig = serde_json::from_str(null_accepted).unwrap();
assert_eq!(config.name, None);
}#[serde(default)] handles missing fields but not null values—null still produces Unexpected::Unit errors.
Practical Examples Summary
use serde::Deserialize;
fn comprehensive_example() {
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct ApiRequest {
id: u64,
name: String,
#[serde(default)]
tags: Vec<String>,
}
// --- Various input scenarios ---
// 1. Valid input - success
let valid = r#"{"id": 1, "name": "test", "tags": ["a", "b"]}"#;
let req: ApiRequest = serde_json::from_str(valid).unwrap();
// 2. Missing optional field - uses default
let missing_optional = r#"{"id": 1, "name": "test"}"#;
let req: ApiRequest = serde_json::from_str(missing_optional).unwrap();
assert!(req.tags.is_empty());
// 3. Null on required field - Unexpected::Unit error
let null_required = r#"{"id": null, "name": "test"}"#;
let err = serde_json::from_str::<ApiRequest>(null_required).unwrap_err();
println!("{}", err); // "invalid type: null, expected u64"
// 4. Unknown field - deny_unknown_fields error
let unknown = r#"{"id": 1, "name": "test", "extra": "value"}"#;
let err = serde_json::from_str::<ApiRequest>(unknown).unwrap_err();
println!("{}", err); // "unknown field `extra`, expected `id`, `name`, or `tags`"
// 5. Unknown field with null - still unknown field error
let unknown_null = r#"{"id": 1, "name": "test", "extra": null}"#;
let err = serde_json::from_str::<ApiRequest>(unknown_null).unwrap_err();
println!("{}", err); // "unknown field `extra`..."
// 6. Missing required field - missing field error
let missing_required = r#"{"id": 1}"#;
let err = serde_json::from_str::<ApiRequest>(missing_required).unwrap_err();
println!("{}", err); // "missing field `name`"
// 7. Type mismatch (string where number expected)
let type_mismatch = r#"{"id": "not a number", "name": "test"}"#;
let err = serde_json::from_str::<ApiRequest>(type_mismatch).unwrap_err();
println!("{}", err); // "invalid type: string \"not a number\", expected u64"
// Error precedence:
// 1. Missing required fields (checked first)
// 2. Unknown fields (if deny_unknown_fields) - field name validation
// 3. Type mismatches (Unexpected::Unit or other Unexpected variants)
}Summary Table
fn summary() {
// ┌────────────────────────────────┬─────────────────────────────────────────────┐
// │ Unexpected::Unit │ │
// ├────────────────────────────────┼─────────────────────────────────────────────┤
// │ What it is │ An enum variant representing null/unit │
// │ When it appears │ When input contains null where a non-unit │
// │ │ type is expected │
// │ Who produces it │ The Deserializer implementation │
// │ What it indicates │ Type mismatch: found null, expected value │
// │ │ │
// │ Example input │ JSON: null │
// │ Example target type │ i32, String, struct, etc. │
// │ Example error message │ "invalid type: null, expected i32" │
// │ │ │
// │ Can be avoided by │ Using Option<T>, custom deserialize impls │
// └────────────────────────────────┴─────────────────────────────────────────────┘
//
// ┌────────────────────────────────┬─────────────────────────────────────────────┐
// │ deny_unknown_fields │ │
// ├────────────────────────────────┼─────────────────────────────────────────────┤
// │ What it is │ A serde attribute for structs/enums │
// │ When it triggers │ When input contains field not defined │
// │ │ in the type │
// │ Who produces it │ Serde derive-generated deserialization │
// │ What it indicates │ Unknown field in input │
// │ │ │
// │ Example input │ {"known": 1, "unknown": 2} │
// │ Example error message │ "unknown field `unknown`, expected `known`"│
// │ │ │
// │ Can be avoided by │ Removing the attribute, using #[serde()] │
│ │ to allow additional fields │
// └────────────────────────────────┴─────────────────────────────────────────────┘
//
// ┌────────────────────────────────┬─────────────────────────────────────────────┐
// │ Processing Order │ │
// ├────────────────────────────────┼─────────────────────────────────────────────┤
// │ 1. Field name recognition │ Is this field defined in the struct? │
// │ │ deny_unknown_fields error occurs here │
// │ │ │
// │ 2. Field presence check │ Are all required fields present? │
// │ │ Missing field error occurs here │
// │ │ │
// │ 3. Field value deserialization │ Does the value match the expected type? │
// │ │ Unexpected::Unit error occurs here │
// │ │ │
// │ Notes: │ │
// │ - Processing stops at first error │
// │ - Unknown field check happens before examining field value │
// │ - Unexpected::Unit is never reached for unknown fields │
// └────────────────────────────────┴─────────────────────────────────────────────┘
//
// ┌────────────────────────────────┬─────────────────────────────────────────────┐
// │ Interaction Scenarios │ │
// ├────────────────────────────────┼─────────────────────────────────────────────┤
// │ Unknown field with null value │ Error: unknown field (field check first) │
// │ Known field with null value │ Error: Unexpected::Unit (type check) │
// │ Optional field with null value │ Success: None (Option accepts null) │
// │ Missing field with default │ Success: default value used │
// │ Missing required field │ Error: missing field │
// │ Unknown field with any value │ Error: unknown field (deny_unknown_fields) │
// └────────────────────────────────┴─────────────────────────────────────────────┘
//
// ┌────────────────────────────────┬─────────────────────────────────────────────┐
// │ Key Distinction │ │
// ├────────────────────────────────┼─────────────────────────────────────────────┤
// │ deny_unknown_fields │ About FIELD NAMES │
// │ │ "Is this field allowed?" │
// │ │ Rejects unrecognized keys │
// │ │ │
// │ Unexpected::Unit │ About FIELD VALUES │
// │ │ "Is this value the right type?" │
// │ │ Reports null where value expected │
// │ │ │
// │ They are ORTHOGONAL │ One operates on field names │
// │ │ One operates on field values │
// │ │ They don't interact directly │
// └────────────────────────────────┴─────────────────────────────────────────────┘
//
// === Key Insight ===
//
// Unexpected::Unit and deny_unknown_fields are independent mechanisms
// that operate at different stages of deserialization:
//
// 1. deny_unknown_fields validates FIELD NAMES
// - Checked before field values are examined
// - Error: "unknown field `name`"
// - Prevents unrecognized keys from reaching your types
//
// 2. Unexpected::Unit validates FIELD VALUES
// - Checked during value deserialization
// - Error: "invalid type: null, expected ..."
// - Reports null/unit where other type expected
//
// The processing order means unknown field errors appear before
// Unexpected::Unit errors, even when the unknown field contains null.
//
// To handle null gracefully:
// - Use Option<T> for nullable fields
// - Use #[serde(default)] for missing fields
// - Use custom deserialize_with for null-as-default
//
// To handle unknown fields:
// - Omit deny_unknown_fields (allows extras silently)
// - Use #[serde(flatten)] to capture extras
// - Keep deny_unknown_fields for strict validation
}Key insight: Unexpected::Unit and deny_unknown_fields operate at fundamentally different levels—deny_unknown_fields validates field names during struct deserialization before examining values, while Unexpected::Unit reports what was found when a type mismatch occurs during value deserialization. They are orthogonal concerns: one guards against unexpected structure (field names), the other describes unexpected content (null values). When both could apply to the same input (unknown field containing null), the unknown field error surfaces first because field name validation precedes value deserialization.