Arc<Mutex>

Guides

Rust walkthroughs

← Back to all guides

How does quote::ToTokens trait enable custom types to be used within quote! macros?

The ToTokens trait is the foundation of the quote! macro's type system—it defines how values convert into token streams that can be interpolated into generated code. When you write quote! { #my_value }, the macro calls ToTokens::to_tokens(&my_value, &mut tokens), allowing the value to write its own representation into the output token stream. This enables custom types to participate in code generation by implementing ToTokens, making them first-class citizens in procedural macros. The trait provides a bridge between runtime values and compile-time token streams, enabling patterns like interpolating syn types directly, converting identifiers, and generating code from structured data.

Basic quote! Usage with Built-in Types

use quote::quote;
 
fn main() {
    let name = "my_function";
    let value = 42u32;
    
    // Built-in types implement ToTokens
    let tokens = quote! {
        fn #name() -> u32 {
            #value
        }
    };
    
    println!("{}", tokens);
    // fn my_function () -> u32 { 42 }
}

quote! interpolates values by calling their ToTokens implementation.

ToTokens Trait Definition

use proc_macro2::TokenStream;
 
// The trait definition (simplified)
pub trait ToTokens {
    fn to_tokens(&self, tokens: &mut TokenStream);
    
    // Provided method for convenience
    fn to_token_stream(&self) -> TokenStream {
        let mut tokens = TokenStream::new();
        self.to_tokens(&mut tokens);
        tokens
    }
}

ToTokens::to_tokens appends tokens to an existing stream for efficiency.

Implementing ToTokens for Custom Types

use proc_macro2::TokenStream;
use quote::{quote, ToTokens, TokenStreamExt};
 
struct StructField {
    name: String,
    ty: String,
}
 
impl ToTokens for StructField {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        let name = &self.name;
        let ty = &self.ty;
        tokens.extend(quote! {
            #name: #ty,
        });
    }
}
 
fn main() {
    let field = StructField {
        name: "count".to_string(),
        ty: "u32".to_string(),
    };
    
    let tokens = quote! {
        struct Counter {
            #field
        }
    };
    
    println!("{}", tokens);
    // struct Counter { count : u32 , }
}

Custom ToTokens implementations define how types become tokens.

Interpolating syn Types

use quote::quote;
use syn::{parse_quote, Ident, Type};
 
fn main() {
    // syn types implement ToTokens
    let ident: Ident = parse_quote!(my_variable);
    let ty: Type = parse_quote!(Vec<String>);
    
    let tokens = quote! {
        let #ident: #ty = Default::default();
    };
    
    println!("{}", tokens);
    // let my_variable : Vec < String > = Default :: default () ;
}

syn types implement ToTokens, enabling direct interpolation.

Using syn Types in Procedural Macros

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput, Ident};
 
#[proc_macro_derive(Builder)]
pub fn derive_builder(input: TokenStream) -> TokenStream {
    let input = parse_macro_input!(input as DeriveInput);
    
    // Extract syn types that implement ToTokens
    let name = &input.ident;
    let vis = &input.vis;
    
    // Interpolate directly into quote!
    let expanded = quote! {
        #vis struct #name {
            inner: String,
        }
        
        impl #name {
            pub fn new() -> Self {
                Self { inner: String::new() }
            }
        }
    };
    
    TokenStream::from(expanded)
}

Procedural macros use ToTokens to generate code from parsed input.

Conditional Code Generation

use quote::quote;
use proc_macro2::TokenStream;
use quote::ToTokens;
 
struct Field {
    name: String,
    has_default: bool,
}
 
impl ToTokens for Field {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        let name = &self.name;
        
        if self.has_default {
            tokens.extend(quote! {
                #name: Default::default(),
            });
        } else {
            tokens.extend(quote! {
                #name,
            });
        }
    }
}
 
fn main() {
    let fields = vec![
        Field { name: "id".to_string(), has_default: false },
        Field { name: "count".to_string(), has_default: true },
    ];
    
    let mut tokens = TokenStream::new();
    for field in &fields {
        field.to_tokens(&mut tokens);
    }
    
    println!("{}", tokens);
}

ToTokens implementations can include conditional logic.

Iterating Over Collections

use quote::quote;
 
fn main() {
    let fields = ["id", "name", "count"];
    
    // Iterators of ToTokens items interpolate correctly
    let tokens = quote! {
        struct Record {
            #(#fields: u32,)*
        }
    };
    
    println!("{}", tokens);
    // struct Record { id : u32 , name : u32 , count : u32 , }
}

quote! handles iteration with #(#var)* syntax.

Complex Repetition Patterns

use quote::quote;
use syn::{parse_quote, Ident};
 
fn main() {
    let field_names: Vec<Ident> = vec![
        parse_quote!(first_name),
        parse_quote!(last_name),
        parse_quote!(age),
    ];
    
    let field_types: Vec<Ident> = vec![
        parse_quote!(String),
        parse_quote!(String),
        parse_quote!(u32),
    ];
    
    // Multiple parallel iterators
    let tokens = quote! {
        struct Person {
            #(#field_names: #field_types,)*
        }
    };
    
    println!("{}", tokens);
    // struct Person { first_name : String , last_name : String , age : u32 , }
}

Multiple collections can be zipped in quote! repetition.

Converting Types to Tokens

use proc_macro2::TokenStream;
use quote::{quote, ToTokens, TokenStreamExt};
 
enum Value {
    Integer(i64),
    String(String),
    Bool(bool),
}
 
impl ToTokens for Value {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        match self {
            Value::Integer(n) => tokens.extend(quote! { #n }),
            Value::String(s) => tokens.extend(quote! { #s }),
            Value::Bool(b) => tokens.extend(quote! { #b }),
        }
    }
}
 
fn main() {
    let values = vec![
        Value::Integer(42),
        Value::String("hello".to_string()),
        Value::Bool(true),
    ];
    
    let tokens = quote! {
        const VALUES: &[&str] = &[#(#values),*];
    };
    
    println!("{}", tokens);
}

Enums can dispatch to appropriate token representations.

TokenStream Extension Methods

use proc_macro2::TokenStream;
use quote::{quote, ToTokens};
 
fn main() {
    let name = "test";
    
    // to_token_stream creates a new TokenStream
    let stream = name.to_token_stream();
    println!("Stream: {}", stream);
    
    // to_tokens appends to existing stream
    let mut combined = TokenStream::new();
    name.to_tokens(&mut combined);
    "another".to_tokens(&mut combined);
    println!("Combined: {}", combined);
}

to_token_stream() creates a new stream; to_tokens() appends to existing.

Identifier Generation

use quote::quote;
use proc_macro2::Ident;
use syn::parse_quote;
 
fn main() {
    let base_name = "field";
    
    // Generate identifiers programmatically
    let idents: Vec<Ident> = (0..3)
        .map(|i| {
            let name = format!("{}_{}", base_name, i);
            // Use quote to create identifier
            quote::format_ident!("{}", name)
        })
        .collect();
    
    let tokens = quote! {
        #(#idents: u32,)*
    };
    
    println!("{}", tokens);
    // field_0 : u32 , field_1 : u32 , field_2 : u32 ,
}

format_ident! creates identifiers with ToTokens support.

Struct Generation with Custom Types

use proc_macro2::TokenStream;
use quote::{quote, ToTokens};
use syn::{parse_quote, Ident, Type};
 
struct Field {
    name: Ident,
    ty: Type,
    is_optional: bool,
}
 
impl ToTokens for Field {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        let name = &self.name;
        let ty = &self.ty;
        
        if self.is_optional {
            tokens.extend(quote! {
                #name: Option<#ty>,
            });
        } else {
            tokens.extend(quote! {
                #name: #ty,
            });
        }
    }
}
 
fn main() {
    let fields = vec![
        Field { name: parse_quote!(id), ty: parse_quote!(u64), is_optional: false },
        Field { name: parse_quote!(name), ty: parse_quote!(String), is_optional: true },
    ];
    
    let tokens = quote! {
        struct Record {
            #(#fields)*
        }
    };
    
    println!("{}", tokens);
    // struct Record { id : u64 , name : Option < String > , }
}

Complex structs can be generated from structured data.

Optional Interpolation

use quote::quote;
use proc_macro2::TokenStream;
use quote::ToTokens;
 
struct MaybeCode {
    code: Option<String>,
}
 
impl ToTokens for MaybeCode {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        if let Some(code) = &self.code {
            // Parse string as tokens
            let parsed: TokenStream = code.parse().unwrap();
            tokens.extend(parsed);
        }
    }
}
 
fn main() {
    let with_code = MaybeCode { code: Some("println!(\"test\")".to_string()) };
    let without_code = MaybeCode { code: None };
    
    let tokens = quote! {
        fn test() {
            #with_code
            #without_code
        }
    };
    
    println!("{}", tokens);
    // fn test () { println ! ("test") }
}

ToTokens can conditionally include or exclude code.

Span Preservation

use proc_macro2::Span;
use quote::quote;
use syn::Ident;
 
fn main() {
    // Spans affect error messages and hygiene
    let span_a = Span::call_site();
    let span_b = Span::call_site();
    
    let ident_a = Ident::new("value", span_a);
    let ident_b = Ident::new("value", span_b);
    
    // Each identifier preserves its span
    let tokens = quote! {
        let #ident_a = #ident_b;
    };
    
    // Spans are preserved through ToTokens
}

ToTokens preserves spans for error reporting and hygiene.

Interpolation in Different Contexts

use quote::quote;
use syn::{parse_quote, Ident};
 
fn main() {
    let name: Ident = parse_quote!(my_field);
    
    // Type position
    let ty_tokens = quote! {
        type MyType = #name;
    };
    
    // Expression position
    let expr_tokens = quote! {
        let x = #name;
    };
    
    // Pattern position
    let pat_tokens = quote! {
        let #name = 42;
    };
    
    // Same ToTokens implementation works in all contexts
    println!("Type: {}", ty_tokens);
    println!("Expr: {}", expr_tokens);
    println!("Pattern: {}", pat_tokens);
}

ToTokens works in any syntactic context.

Custom Derive Macro Example

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput, FieldsNamed, Ident};
 
#[proc_macro_derive(Builder)]
pub fn derive_builder(input: TokenStream) -> TokenStream {
    let input = parse_macro_input!(input as DeriveInput);
    let name = &input.ident;
    
    // Generate builder struct name
    let builder_name = quote::format_ident!("{}Builder", name);
    
    // Get fields if it's a struct with named fields
    let fields = match &input.data {
        syn::Data::Struct(syn::DataStruct {
            fields: syn::Fields::Named(fields),
            ..
        }) => &fields.named,
        _ => panic!("Only named field structs are supported"),
    };
    
    // Field names implement ToTokens (they're syn::Ident)
    let field_names: Vec<&Ident> = fields.iter()
        .map(|f| &f.ident)
        .flatten()
        .collect();
    
    let expanded = quote! {
        pub struct #builder_name {
            #(#field_names: Option<String>,)*
        }
        
        impl #builder_name {
            pub fn build(self) -> #name {
                #name {
                    #(#field_names: self.#field_names.unwrap_or_default(),)*
                }
            }
        }
    };
    
    TokenStream::from(expanded)
}

Complete derive macro using ToTokens for all syn types.

Nested Token Generation

use proc_macro2::TokenStream;
use quote::{quote, ToTokens};
 
struct Attribute {
    name: String,
    args: Vec<String>,
}
 
impl ToTokens for Attribute {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        let name = &self.name;
        let args = &self.args;
        
        tokens.extend(quote! {
            #[#name(#(#args),*)]
        });
    }
}
 
struct Struct {
    name: String,
    attrs: Vec<Attribute>,
    fields: Vec<String>,
}
 
impl ToTokens for Struct {
    fn to_tokens(&self, tokens: &mut TokenStream) {
        let name = &self.name;
        let attrs = &self.attrs;
        let fields = &self.fields;
        
        tokens.extend(quote! {
            #(#attrs)*
            struct #name {
                #(#fields: u32,)*
            }
        });
    }
}

Composing ToTokens implementations builds complex generated code.

String vs Token Interpolation

use quote::quote;
use proc_macro2::TokenStream;
 
fn main() {
    let name = "my_function";
    
    // Interpolated as identifier (not string literal)
    let tokens = quote! {
        fn #name() {}
    };
    println!("As ident: {}", tokens);
    // fn my_function () { }
    
    // To create string literal, use stringify or quote
    let tokens2 = quote! {
        const NAME: &str = stringify!(#name);
    };
    println!("As string: {}", tokens2);
    // const NAME : & str = stringify ! (my_function) ;
}

Strings interpolate as identifiers; use stringify! for string literals.

Comparison Table

Method Purpose Returns
to_tokens(&mut stream) Append tokens to stream ()
to_token_stream() Create new stream TokenStream
quote! { #val } Interpolate value TokenStream

Synthesis

ToTokens is the connective tissue between Rust values and generated code:

How it works: When you write #value inside quote!, the macro calls value.to_tokens(&mut tokens). The value's ToTokens implementation decides what tokens to write. For syn types like Ident, Type, and Expr, the implementation writes the parsed representation. For strings, it writes the string content as an identifier. For primitives, it writes their literal representation.

Custom implementations: By implementing ToTokens for your own types, you control how they appear in generated code. This enables structured code generation where types represent code fragments—fields, attributes, entire function bodies—and compose naturally within quote! blocks.

Key pattern: In procedural macros, parse input with syn, manipulate the structured representation, and interpolate the result with quote!. The ToTokens implementations for syn types handle the token-level details, letting you work at a higher abstraction level.

Efficiency: The to_tokens(&mut TokenStream) signature allows appending to existing streams without allocation for each interpolation, making quote! efficient for generating large code blocks. The alternative to_token_stream() method creates a new stream when you need ownership.