Loading pageā¦
Rust walkthroughs
Loading pageā¦
Pin for self-referential futures and how does std::pin::pin! macro help?Pin is a wrapper type that prevents moving a value after it has been pinned, which is essential for self-referential futures generated by async code. When an async function awaits another future, the generated state machine may contain references to its own fieldsācreating self-referential structures that would break if the future were moved in memory. The Pin<P> type constrains the pointer type P (such as &mut T or Box<T>) to guarantee the pointed-to value won't be moved, allowing safe self-references. The std::pin::pin! macro creates a Pin<&mut T> on the stack, handling the pinning ceremony automatically and ensuring the future can't be moved after polling begins.
// This is what async transforms into conceptually
// An async function like:
async fn example() {
let data = String::from("hello");
let reference = &data; // Reference to local variable
other_async().await; // Yield point
println!("{}", reference); // Use reference after yield
}
// Gets transformed into a state machine like:
enum ExampleFuture {
State0 { data: String },
State1 { data: String, reference: &String, other_future: OtherFuture },
Complete,
}
// In State1, `reference` points to `data` which is in the SAME struct
// If this state machine were MOVED, `data` would be at a new address
// but `reference` would still point to the OLD address -> UB!Self-referential structures cannot be safely moved after creation because internal references would dangle.
struct SelfReferential {
data: String,
pointer: *const String, // Points to `data` field
}
impl SelfReferential {
fn new() -> Self {
let mut this = SelfReferential {
data: String::from("hello"),
pointer: std::ptr::null(),
};
this.pointer = &this.data as *const String; // Points into self
this
}
}
fn demonstrate_problem() {
let s1 = SelfReferential::new();
let s2 = s1; // Move!
// s1.data has moved to a new address
// s2.pointer still points to s1's old data address
// Dereferencing s2.pointer is UB!
}
// Rust prevents this with Pin - the type system encodes the "don't move" constraintMoving self-referential structures invalidates internal pointers, causing undefined behavior.
use std::pin::Pin;
// Pin wraps a pointer type
// Pin<&mut T> - pinned mutable reference
// Pin<Box<T>> - pinned box (heap allocation)
// The key guarantee: once pinned, you CANNOT move the value out
// You can only get &T or &mut T (if T: Unpin)
// Unpin trait: types that are safe to move even when pinned
// Most types are Unpin by default
// Self-referential futures do NOT implement Unpin
fn pin_basics() {
let mut value = 42;
// Cannot create Pin<&mut T> directly - must use safe APIs
// This would be unsafe:
// let pinned: Pin<&mut i32> = unsafe { Pin::new_unchecked(&mut value) };
// Safe way with Box:
let pinned: Pin<Box<i32>> = Box::pin(42);
// i32 is Unpin, so we can get &mut even when pinned
let inner: &mut i32 = pinned.get_mut();
*inner = 100;
// For !Unpin types, you cannot get &mut T safely
}Pin wraps pointer types to provide move-prevention guarantees.
use std::pin::Pin;
use std::marker::Unpin;
// Unpin means "it's okay to move this type even when pinned"
// Most types automatically implement Unpin
struct RegularStruct {
data: String,
count: i32,
}
// RegularStruct is Unpin by default
struct SelfRefStruct {
data: String,
pointer: *const String, // Self-referential
}
// SelfRefStruct is NOT Unpin (would need manual impl, but shouldn't)
// !Unpin types cannot be moved after pinning
fn unpin_demo() {
let mut regular = Box::pin(RegularStruct {
data: "hello".to_string(),
count: 0,
});
// RegularStruct is Unpin, so we can get mutable reference
let inner: &mut RegularStruct = regular.get_mut();
inner.count += 1;
// SelfRefStruct would NOT allow get_mut (it's !Unpin)
// Only way to get &mut would be unsafe code
}Unpin marks types that are safe to move. Self-referential futures are !Unpin.
use std::pin::pin;
use std::future::Future;
async fn my_async_function() -> i32 {
std::future::ready(42).await
}
fn using_pin_macro() {
// pin! creates a pinned future on the stack
let mut future = pin!(my_async_function());
// The macro:
// 1. Creates a local variable holding the future
// 2. Creates a Pin<&mut T> pointing to it
// 3. Ensures the future can't be moved after
// Now we can poll the future
use std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker};
// Create a no-op waker for demonstration
fn dummy_raw_waker() -> RawWaker {
fn no_op(_: *const ()) {}
fn clone(_: *const ()) -> RawWaker { dummy_raw_waker() }
static VTABLE: RawWakerVTable = RawWakerVTable::new(clone, no_op, no_op, no_op);
RawWaker::new(std::ptr::null(), &VTABLE)
}
let waker = unsafe { Waker::from_raw(dummy_raw_waker()) };
let mut cx = Context::from_waker(&waker);
match future.as_mut().poll(&mut cx) {
Poll::Ready(result) => println!("Result: {}", result),
Poll::Pending => println!("Pending"),
}
}pin! safely creates pinned references on the stack.
use std::pin::Pin;
use std::future::Future;
async fn my_async_function() -> i32 {
std::future::ready(42).await
}
fn manual_pinning() {
// Without pin! macro, you'd need:
// Option 1: Box::pin (heap allocation)
let mut future: Pin<Box<dyn Future<Output = i32>>> = Box::pin(my_async_function());
// Option 2: unsafe stack pinning (requires understanding guarantees)
let mut future = my_async_function();
let mut future: Pin<&mut _> = unsafe {
// SAFETY: future is not moved after this point
Pin::new_unchecked(&mut future)
};
// pin! macro does Option 2 safely
// It uses a special trick to prevent moving:
// Conceptually, pin!(expr) expands to something like:
// let mut future = expr;
// let mut future = unsafe { Pin::new_unchecked(&mut future) };
// But with additional static checks
}Before pin!, you needed Box::pin (heap overhead) or unsafe (error-prone).
use std::pin::pin;
fn how_pin_works() {
// The pin! macro works by:
// 1. Creating a local variable
// 2. Immediately creating a Pin<&mut T> to it
// 3. Structured so the reference can't outlive the variable
// This is similar to:
fn pin_pattern<T>(mut value: T) -> Pin<&mut T> {
// The value is moved INTO this function
// Then we create a pinned reference
// The reference is tied to the local `value`
// value cannot be moved because we have a reference to it
unsafe { Pin::new_unchecked(&mut value) }
// But this doesn't quite work - value would be dropped!
// The real macro uses a more sophisticated approach
}
// pin! uses the "stack pinning" pattern:
let mut pinned = pin!(my_async_func());
// The macro ensures:
// - The future is created in place
// - A Pin<&mut Future> is created pointing to it
// - The future cannot be moved (no way to access it unpinned)
}
async fn my_async_func() {}The macro uses Rust's borrowing rules to statically prevent moves after pinning.
use std::pin::Pin;
use std::future::Future;
use std::task::{Context, Poll};
// A future that might be self-referential (simplified)
struct MyFuture {
data: String,
state: usize,
// Imagine there's an implicit self-reference after the first poll
}
impl Future for MyFuture {
type Output = String;
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
// The self: Pin<&mut Self> signature means:
// - We have a pinned mutable reference
// - We CANNOT move self
// - Self-referential fields are safe because we're pinned
// We can access fields through the pinned reference
let this = self.get_mut(); // Safe because we don't move anything
match this.state {
0 => {
this.state = 1;
Poll::Pending
}
1 => {
this.state = 2;
Poll::Pending
}
_ => {
Poll::Ready(this.data.clone())
}
}
}
}
fn use_self_referential_future() {
let future = MyFuture {
data: "hello".to_string(),
state: 0,
};
// Must pin before polling
let mut pinned = Pin::new(Box::new(future));
// or: let mut pinned = pin!(future); (on stack)
// Now we can poll safely
}Pin<&mut Self> in poll ensures self-references remain valid.
use std::pin::pin;
use std::future::Future;
async fn self_referential_example() -> i32 {
let data = vec![1, 2, 3, 4, 5];
let reference = &data; // Reference into local variable
some_async_op().await; // Yield point - state saved
// After resuming, `data` must still be at same address
// Otherwise `reference` would dangle
reference.len() as i32
}
async fn some_async_op() {
// Some async operation
}
fn why_pinning() {
// This async function creates a self-referential future
let future = self_referential_example();
// Without pinning, the future could be moved:
// let future2 = future; // This would invalidate internal references!
// pin! prevents this:
let mut pinned = pin!(self_referential_example());
// Now the future is pinned - it can't be moved
// Self-references inside are safe
}
// The async transformation creates a state machine enum:
// enum SelfRefFuture {
// State0 { data: Vec<i32> },
// State1 { data: Vec<i32>, reference: &Vec<i32>, inner: SomeAsyncOp },
// }
//
// In State1, `reference` points to `data` - a self-reference!
// Moving this enum would break the referenceAsync functions can create self-referential futures that must not be moved.
use std::pin::{Pin, pin};
async fn my_async() -> i32 { 42 }
fn pinning_comparison() {
// Stack pinning with pin! macro
{
let mut future = pin!(my_async());
// Future is on the stack
// No heap allocation
// Must be used within this scope
// Can't return from function
let _: Pin<&mut dyn Future<Output = i32>> = future.as_mut();
}
// Heap pinning with Box::pin
{
let future: Pin<Box<dyn Future<Output = i32>>> = Box::pin(my_async());
// Future is on the heap
// Can be returned from function
// Can be stored in collections
// Slight allocation overhead
let _: Pin<Box<dyn Future<Output = i32>>> = future;
}
// Choose based on use case:
// - Stack pinning for local async blocks, await points
// - Heap pinning for storing futures, returning from functions
}Stack pinning (pin!) is cheaper; heap pinning (Box::pin) is more flexible.
use std::pin::pin;
use std::future::{Future, poll_fn};
use std::task::{Context, Poll};
// Common pattern: polling a future manually
fn poll_future_manually() {
let mut future = pin!(my_async());
// Create a simple waker
let waker = futures::task::noop_waker();
let mut cx = Context::from_waker(&waker);
loop {
match future.as_mut().poll(&mut cx) {
Poll::Ready(result) => {
println!("Completed with: {}", result);
break;
}
Poll::Pending => {
println!("Still pending...");
// In real code, would wait for wake notification
break;
}
}
}
}
async fn my_async() -> i32 {
std::future::ready(42).await
}
// Another pattern: implementing a future combinator
struct Map<F, T, Fut>
where
Fut: Future<Output = T>,
{
future: Fut,
f: Option<F>,
}
impl<F, T, Fut> Future for Map<F, T, Fut>
where
Fut: Future<Output = T>,
F: FnOnce(T) -> T,
{
type Output = T;
// Note: self: Pin<&mut Self> - we receive pinned self
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<T> {
// We must project the pin to inner fields safely
let this = self.get_mut(); // Safe for our simple case
// Poll the inner future
let mut future = Pin::new(&mut this.future);
match future.as_mut().poll(cx) {
Poll::Ready(value) => {
let f = this.f.take().unwrap();
Poll::Ready(f(value))
}
Poll::Pending => Poll::Pending,
}
}
}Futures receive Pin<&mut Self> and must safely project pins to inner fields.
use std::pin::Pin;
use std::marker::PhantomPinned;
// Struct with !Unpin field
struct SelfRefFuture {
data: String,
pointer: *const String, // Self-referential
_pinned: PhantomPinned, // Makes struct !Unpin
}
impl SelfRefFuture {
// Safe pin projection for struct with !Unpin fields
fn get_data(self: Pin<&mut Self>) -> Pin<&mut String> {
// SAFETY: We're projecting the pin to a field
// The field is structurally pinned - if Self can't move, field can't move
unsafe { self.map_unchecked_mut(|s| &mut s.data) }
}
fn get_pointer(self: Pin<&Self>) -> *const String {
self.pointer
}
}
// For structs with only Unpin fields, projection is simpler
struct UnpinStruct {
data: String,
count: i32,
}
impl UnpinStruct {
// Since UnpinStruct is Unpin, we can get &mut directly
fn get_data_mut(self: Pin<&mut Self>) -> &mut String {
// SAFE because UnpinStruct: Unpin
self.get_mut().data = "new".to_string();
&mut self.get_mut().data
}
}Pin projection safely propagates pinning guarantees to inner fields.
use std::pin::{Pin, pin};
use std::future::Future;
// Pitfall 1: Trying to move out of Pin
fn move_out_of_pin() {
let pinned = pin!(String::from("hello"));
// This WON'T COMPILE - can't move out of Pin
// let s: String = *pinned; // Error!
// For Unpin types, you can get_mut first
// But that's not allowed for !Unpin futures
}
// Pitfall 2: Creating invalid self-references
fn invalid_self_ref() {
struct Bad {
data: String,
reference: &'static String, // &'static can't point to local
}
// This won't compile - can't create self-reference with 'static lifetime
}
// Pitfall 3: Forgetting to pin before polling
fn forget_to_pin() {
let future = my_async();
// This WON'T COMPILE - poll requires Pin<&mut Self>
// future.poll(&mut cx); // Error: future.poll() doesn't exist
// Must pin first:
let mut pinned = pin!(future);
// Now we can poll (but usually we just .await)
}
async fn my_async() {}
// Correct pattern: always pin before polling
fn correct_pattern() {
let mut future = pin!(my_async());
// Now we have Pin<&mut impl Future>
// Can call poll() or use other pinned APIs
}The type system prevents common mistakes with Pin.
| Concept | Purpose |
|---------|---------|
| Pin<P> | Wrapper preventing moves through pointer P |
| !Unpin | Marker that type cannot be moved once pinned |
| PhantomPinned | Make a struct !Unpin even if all fields are Unpin |
| pin! macro | Safe stack pinning without heap allocation |
| Box::pin | Heap pinning with allocation |
| Pin::new_unchecked | Unsafe pin creation (requires manual guarantees) |
| Pin projection | Propagating pin guarantees to inner fields |
Pin solves the self-referential future problem by encoding move prevention in the type system:
The problem: Async functions create state machines that may contain self-references (references to local variables that persist across .await points). If such a future were moved, internal references would dangle.
The solution: Pin<&mut T> guarantees the pointed-to T won't be moved, making self-references safe. For !Unpin types (like self-referential futures), you cannot get &mut T from Pin<&mut T> safely, preventing code from moving the value.
The pin! macro: Creates a Pin<&mut T> on the stack without heap allocation. It:
Key insight: Pin is a contractāpinning a value promises "I won't move this." For Unpin types (most types), this is trivial because they don't have self-references. For !Unpin types (futures with self-references), the pin promise is essential for safety. The pin! macro is the ergonomic way to make this promise safely on the stack, while Box::pin does the same on the heap. Both ensure the future can be safely polled without risking dangling references inside its state machine.