Stabilize derive(CoercePointee)

[dingxiangfei2009]

What is being proposed for stabilization

This stabilization report concerns the macro specified by RFC3621, now called CoercePointee.

This macro enables the derivation of trait implementation of Unsize, CoerceUnsize and DispatchFromDyn traits, both of which are unstable at the moment, under a well-defined and structured schema, so that the users who would like to allow their custom type to admit unsizing coercion and use of dyn DST in one of its generic type parameter in a safe way. The greatest use case of this macro is to allow users to equip their types with this behaviour without dependence on alloc crate, or when alloc is not available and a custom implementation of pointer-like data structure in the likes of Rc and Arc is highly desirable. The most prominent example is the kernel crate by Rust-for-Linux, which would greatly benefit from this macro due to reduction of boilerplate, and reduce the project's dependence on unstable features behind Unsize, CoerceUnsize and DispatchFromDyn.

In a nutshell, derive(CoercePointee) derives the implementation of CoerceUnsize and DispatchFromDyn traits on the target struct definition with all the necessary trait bounds. It identifies one generic type variable in the generic list of the target struct, that is either uniquely annotated with the #[pointee] attribute or is the only type variable among the generics. This identified generic, which is called source type in this document, will be treated as the target of unsizing operation and dyn trait object dispatch. Correspondingly, the resultant type after the unsizing coercion will be called target type in this document. In case additional trait bounds applies on the source type, the said trait bounds will be migrated to both the source type and the target type in the trait implementation.

Out of necessity of unsizing coercion, the macro requires the struct to adopt the repr(transparent) layout. The only data field in the struct definition, which is required by this layout, should also implement CoerceUnsize and DispatchFromDyn at the moment, or at least conform to requirement of the unsizing coercion and trait object dispatching protocol.

As an example, here is how the generated code would look like after expanding the macro. The following is a user source that invokes this macro.

pub trait MyTrait<T: ?Sized> {}

#[derive(core::marker::CoercePointee)]
#[repr(transparent)]
pub struct MyPointer2<'a, Y, Z: MyTrait<T>, #[pointee] T: ?Sized + MyTrait<T>, X: MyTrait<T> = ()>
where
    Y: MyTrait<T>,
{
    data: &'a mut T,
    x: core::marker::PhantomData<X>,
}

This is the expansion result. The source type T after the unsizing coercion is assumed to be __S and the trait bounds requested by the original definition are migrated for __S as well throughtout.

#[repr(transparent)]
pub struct MyPointer2<
    'a, Y, Z: MyTrait<T>,
    #[pointee] T: ?Sized + MyTrait<T>,
    X: MyTrait<T> = ()
>
where Y: MyTrait<T>
{
    data: &'a mut T,
    x: core::marker::PhantomData<X>,
}

#[automatically_derived]
impl<
    'a, Y,
    Z: MyTrait<T> + MyTrait<__S>,
    T: ?Sized + MyTrait<T> + ::core::marker::Unsize<__S>,
    __S: ?Sized + MyTrait<__S>,
    X: MyTrait<T> + MyTrait<__S>
>
::core::ops::DispatchFromDyn<MyPointer2<'a, Y, Z, __S, X>> for MyPointer2<'a, Y, Z, T, X>
where Y: MyTrait<T>, Y: MyTrait<__S> {
}

#[automatically_derived]
impl<
    'a, Y,
    Z: MyTrait<T> + MyTrait<__S>,
    T: ?Sized + MyTrait<T> + ::core::marker::Unsize<__S>,
    __S: ?Sized + MyTrait<__S>,
    X: MyTrait<T> + MyTrait<__S>
>
::core::ops::CoerceUnsized<MyPointer2<'a, Y, Z, __S, X>> for MyPointer2<'a, Y, Z, T, X>
where Y: MyTrait<T>, Y: MyTrait<__S> {
}

Future interaction

This stabilization of this macro does not require the stabilization of any of the unstable traits mentioned. This macro aims at stabilising a subset of features thereof, which has widely proliferated in the ecosystem, without blocking further development of DSTs and unsizing coercion. In case of change in the design of these language features, as long as basic provision of the coercion operation remains the same, the macro should be updated to use the new facility inside its implementation when necessary, to preserve only the semantics and capabilities that this macro exposes users to.

Tracking:

• Tracking issue for RFC 3621: derive(CoercePointee) #123430

cc @rust-lang/lang

[rustbot]

r? @joboet

rustbot has assigned @joboet.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[dingxiangfei2009]

@rustbot ready

• Documentation is updated to reflect the requirements as laid out by the RFC

[dingxiangfei2009]

@rustbot label +F-derive_smart_pointer

[dingxiangfei2009]

Question:

Do we need to condition on bootstrap? No standard library item depends on this macro yet.

[Darksonn]

The RFC for this feature said that it depends on the arbitrary self types feature, but it is not a full blocker to stabilizing this feature. There are two things that the macro enables:

The first thing is coercions from MySmartPtr<MyType> to MySmartPtr<dyn MyTrait>.

The second thing is making traits such as this one object safe:

trait MyTrait {
    fn func(self: MySmartPtr<Self>);
}

Without arbitrary self types, it's not legal to declare this trait because MySmartPtr is not a valid self type. However, the coercions from MySmartPtr<MyType> to MySmartPtr<dyn MyTrait> still work, and custom smart pointers are still useful, e.g. given an MyArc<dyn MyTrait> that derefs to &dyn MyTrait, you can call trait methods on MyTrait that take &self just fine.

So if we stabilize the macro now, then we gain the ability to make coercions and call trait methods via deref, but we don't gain the ability to declare traits that take the smart pointer without a deref coercion. We would gain that ability once arbitrary self types are stabilized.

[traviscross]

@rfcbot fcp merge

Thanks @dingxiangfei2009. We talked about this in our triage call today. Let's ship it.

[rfcbot]

Team member @traviscross has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[nikomatsakis]

@compiler-errors my preferred way to manage that is not unchecking my box but filing a concern

[compiler-errors]

Yeah, at this point @lcnr has already begun to do a good job at characterizing the types-level surface of what is being stabilized here, and funny enough that they found an interesting quirk in the process :D

[BoxyUwU]

@rfcbot concern not properly checking repr(transparent) #135206

[BoxyUwU]

DispatchFromDyn doesn't require the type parameter to only be used in one field and potentially phantomdata, instead it requires one field and arbitarily many ZSTs playground. This in theory means that dispatch from dyn impls can result in effectively arbitrary transmutes from MyZST<dyn Trait> to MyZST<Foo>.

In practice I don't think this is exploitable, regardless I don't think we should stabilize the ability for semi-user-written impls or trait bounds to be written involving this trait until the builtin checks are brought in line with CoerceUnsized.

edit: thanks to @steffahn for pointing this out

@rfcbot concern DispatchFromDyn builtin checks are weaker than CoerceUnsized

[BoxyUwU]

The CoerceUnsized builtin checks that the parameter being unsized is only used in PhantomDatas and one field does not handle aliases and just directly checks if the Ty is a PhantomData. This will have to be fixed before stabilizing lazy_type_alias so we may as well just handle it from the beginning. See the following two playground examples for things that don't compile but probably ought to before we stabilize any form of user written impls for the trait:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=35e9d30ae8a278601de5d8a45bc40417
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=850f363579a53b61b88c6b02a502c31b

@rfcbot concern CoerceUnsized builtin checks do not handle aliases

[BoxyUwU]

I've moved some of the concerns out to separate issues where people can assign themselves and Fixes #... them

[bors]

☔ The latest upstream changes (presumably #135286) made this pull request unmergeable. Please resolve the merge conflicts.

[BoxyUwU]

@rfcbot resolved DispatchFromDyn builtin checks are weaker than CoerceUnsized
@rfcbot resolved CoerceUnsized builtin checks do not handle aliases

[BoxyUwU]

I dont know who the "owners" of arbitrary self types are but I would like for them to sign off on this to some degree as this does stabilize additional method receivers and I can't speak to the status of the implementation of that feature.

@rfcbot concern arbitrary self types impl status

From a types POV though I've spent a while thinking over the traits involved and feel somewhat convinced that DispatchFromDyn is conceptually fine especially with the restrictions imposed by CoercePointee. I will write something more about this on the tracking issue though, or here later as other people might find the reasoning useful

[BoxyUwU]

cc @adetaylor from the tracking issues of arbitrary self types it seems like you've been rather with arbitrary self types? see the above comment

[adetaylor]

Yep, I've been the one driving arbitrary self types. I'm in the early stages of drafting a stabilization report for it - hopefully I'll have written that up in about a week, but I've got a few experiments I need to run first, so it might be a wee bit longer. I'll read through here as well in the next few days and add any comments/questions I've got.

[Darksonn]

as this does stabilize additional method receivers and I can't speak to the status of the implementation of that feature.

This feature doesn't have any effect on what method receivers are possible. Adding #[derive(CoercePointee)] to MySmartPointer changes whether traits containing self: MySmartPointer<Self> are object safe or not if it's legal to declare such a trait. However, the macro has no influence on whether it's legal to declare such a trait.

[BoxyUwU]

Ah okay, I did not realise that dispatch from dyn was completely irrelevant to this stabilization given that it had been previously talked about and calling trait methods on these smart pointers had been thrown around in code examples (and the description of the PR explicitly showing that DispatchFromDyn had been derived)

[BoxyUwU]

@rfcbot resolved concern arbitrary self types impl status

[RalfJung]

This is about object safety, so doesn't that mean that DispatchFromDyn is indeed relevant?

[steffahn]

@RalfJung Current dyn-compatibility definitions also require all (non-Self: Sized) trait methods to have a self argument, and dispatch to happen through the self argument. Otherwise, it’s never dyn-compatible anyway.

This means that without arbitrary_self_types, the question of whether DispatchFromDyn is implemented or not is only relevant for types supported as self-types anyway:

If the type of the self parameter is specified, it is limited to types resolving to one generated by the following grammar (where 'lt denotes some arbitrary lifetime):

P = &'lt S | &'lt mut S | Box<S> | Rc<S> | Arc<S> | Pin<P>
S = Self | P

(according to the reference)

So custom smart pointers are never relevant. (The type Self becoming a custom smart pointer in trait impls is irrelevant, too, because dyn-compatibility is always determined in the abstract, so Self has the abstract meaning of “the formal Self parameter”, not a synonym of a specific type.)

[Darksonn]

I mean, arbitrary self types are relevant because you cannot use the DispatchFromDyn part of #[derive(CoercePointee)] unless you can declare a trait with self: MySmartPtr<Self>, which you can't do without arbitrary self types. But you can still do this without arbitrary self types:

trait MyTrait {
    fn foo(&mut self);
}

fn foo(arc: MySmartPtr<dyn MyTrait>) {
    arc.foo();
}

foo(MySmartPtr::new(MyStruct::new()));

[joboet]

I'm not the right reviewer for this (just cleaning up my review queue, this is waiting on FCP anyway).
r? types

[joboet]

@rustbot ping
Does this bot dream of electronic sheep?