Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

require #[pyclass] to be Sync #4566

Merged
merged 15 commits into from
Nov 5, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions examples/decorator/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};
use std::cell::Cell;
use std::sync::atomic::{AtomicU64, Ordering};

/// A function decorator that keeps track how often it is called.
///
Expand All @@ -10,7 +10,7 @@ pub struct PyCounter {
// Keeps track of how many calls have gone through.
//
// See the discussion at the end for why `Cell` is used.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please update (or maybe delete) this line and the discussion it references? I'm fine with docs being added in followup PRs but it's bad form to leave docs in a wrong (rather than just incomplete or missing) state.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks for pointing out that I missed that.

count: Cell<u64>,
count: AtomicU64,

// This is the actual function being wrapped.
wraps: Py<PyAny>,
Expand All @@ -26,14 +26,14 @@ impl PyCounter {
#[new]
fn __new__(wraps: Py<PyAny>) -> Self {
PyCounter {
count: Cell::new(0),
count: AtomicU64::new(0),
wraps,
}
}

#[getter]
fn count(&self) -> u64 {
self.count.get()
self.count.load(Ordering::Relaxed)
}

#[pyo3(signature = (*args, **kwargs))]
Expand All @@ -43,9 +43,7 @@ impl PyCounter {
args: &Bound<'_, PyTuple>,
kwargs: Option<&Bound<'_, PyDict>>,
) -> PyResult<Py<PyAny>> {
let old_count = self.count.get();
let new_count = old_count + 1;
self.count.set(new_count);
let new_count = self.count.fetch_add(1, Ordering::Relaxed);
let name = self.wraps.getattr(py, "__name__")?;

println!("{} has been called {} time(s).", name, new_count);
Expand Down
8 changes: 5 additions & 3 deletions guide/src/class/protocols.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,18 +158,20 @@ Example:
```rust
use pyo3::prelude::*;

use std::sync::Mutex;

#[pyclass]
struct MyIterator {
iter: Box<dyn Iterator<Item = PyObject> + Send>,
iter: Mutex<Box<dyn Iterator<Item = PyObject> + Send>>,
}

#[pymethods]
impl MyIterator {
fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> {
slf
}
fn __next__(mut slf: PyRefMut<'_, Self>) -> Option<PyObject> {
slf.iter.next()
fn __next__(slf: PyRefMut<'_, Self>) -> Option<PyObject> {
slf.iter.lock().unwrap().next()
}
}
```
Expand Down
2 changes: 1 addition & 1 deletion guide/src/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1800,7 +1800,7 @@ There can be two fixes:
```

After:
```rust
```rust,ignore
# #![allow(dead_code)]
use pyo3::prelude::*;
use std::sync::{Arc, Mutex};
Expand Down
5 changes: 5 additions & 0 deletions newsfragments/4566.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
* The `pyclass` macro now creates a rust type that is `Sync` by default. If you
would like to opt out of this, annotate your class with
`pyclass(unsendable)`. See the migraiton guide entry (INSERT GUIDE LINK HERE)
for more information on updating to accommadate this change.

14 changes: 14 additions & 0 deletions pyo3-macros-backend/src/pyclass.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2310,7 +2310,21 @@ impl<'a> PyClassImplsBuilder<'a> {
}
});

let assertions = if attr.options.unsendable.is_some() {
TokenStream::new()
} else {
quote_spanned! {
cls.span() =>
const _: () = {
use #pyo3_path::impl_::pyclass::*;
assert_pyclass_sync::<#cls, { IsSync::<#cls>::VALUE }>();
};
}
};

Ok(quote! {
#assertions

#pyclass_base_type_impl

impl #pyo3_path::impl_::pyclass::PyClassImpl for #cls {
Expand Down
4 changes: 4 additions & 0 deletions src/coroutine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ pub struct Coroutine {
waker: Option<Arc<AsyncioWaker>>,
}

// Safety: `Coroutine` is allowed to be `Sync` even though the future is not,
// because the future is polled with `&mut self` receiver
unsafe impl Sync for Coroutine {}
Comment on lines +38 to +40
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same logic that Exclusive uses? If so, I would prefer that we use that (or our own version of it) rather than this impl.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I marked this ready for review, but I think this hasn't been looked at yet. I'll leave that to David since I'm not sure if using Exclusive makes sense.


impl Coroutine {
/// Wrap a future into a Python coroutine.
///
Expand Down
66 changes: 5 additions & 61 deletions src/impl_/pyclass.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,13 @@ use std::{
thread,
};

mod assertions;
mod lazy_type_object;
mod probes;

pub use assertions::*;
pub use lazy_type_object::LazyTypeObject;
pub use probes::*;

/// Gets the offset of the dictionary from the start of the object in bytes.
#[inline]
Expand Down Expand Up @@ -1418,67 +1423,6 @@ impl<ClassT: PyClass, FieldT, Offset: OffsetCalculator<ClassT, FieldT>>
}
}

/// Trait used to combine with zero-sized types to calculate at compile time
/// some property of a type.
///
/// The trick uses the fact that an associated constant has higher priority
/// than a trait constant, so we can use the trait to define the false case.
///
/// The true case is defined in the zero-sized type's impl block, which is
/// gated on some property like trait bound or only being implemented
/// for fixed concrete types.
pub trait Probe {
const VALUE: bool = false;
}

macro_rules! probe {
($name:ident) => {
pub struct $name<T>(PhantomData<T>);
impl<T> Probe for $name<T> {}
};
}

probe!(IsPyT);

impl<T> IsPyT<Py<T>> {
pub const VALUE: bool = true;
}

probe!(IsToPyObject);

#[allow(deprecated)]
impl<T: ToPyObject> IsToPyObject<T> {
pub const VALUE: bool = true;
}

probe!(IsIntoPy);

#[allow(deprecated)]
impl<T: IntoPy<crate::PyObject>> IsIntoPy<T> {
pub const VALUE: bool = true;
}

probe!(IsIntoPyObjectRef);

// Possible clippy beta regression,
// see https://github.com/rust-lang/rust-clippy/issues/13578
#[allow(clippy::extra_unused_lifetimes)]
impl<'a, 'py, T: 'a> IsIntoPyObjectRef<T>
where
&'a T: IntoPyObject<'py>,
{
pub const VALUE: bool = true;
}

probe!(IsIntoPyObject);

impl<'py, T> IsIntoPyObject<T>
where
T: IntoPyObject<'py>,
{
pub const VALUE: bool = true;
}

/// ensures `obj` is not mutably aliased
#[inline]
unsafe fn ensure_no_mutable_alias<'py, ClassT: PyClass>(
Expand Down
52 changes: 52 additions & 0 deletions src/impl_/pyclass/assertions.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/// Helper function that can be used at compile time to emit a diagnostic if
/// the type does not implement `Sync` when it should.
///
/// The mere act of invoking this function will cause the diagnostic to be
/// emitted if `T` does not implement `Sync` when it should.
///
/// The additional `const IS_SYNC: bool` parameter is used to allow the custom
/// diagnostic to be emitted; if `PyClassSync`
pub const fn assert_pyclass_sync<T, const IS_SYNC: bool>()
where
T: PyClassSync<IS_SYNC> + Sync,
{
}

#[cfg_attr(
diagnostic_namespace,
diagnostic::on_unimplemented(
message = "the trait `Sync` is not implemented for `{Self}`",
label = "required by `#[pyclass]`",
note = "replace thread-unsafe fields with thread-safe alternatives",
note = "see <TODO INSERT PYO3 GUIDE> for more information",
)
)]
pub trait PyClassSync<const IS_SYNC: bool>: private::Sealed<IS_SYNC> {}

mod private {
pub trait Sealed<const IS_SYNC: bool> {}
impl<T> Sealed<true> for T {}
#[cfg(not(diagnostic_namespace))]
impl<T> Sealed<false> for T {}
}

// If `true` is passed for the const parameter, then the diagnostic will
// not be emitted.
impl<T> PyClassSync<true> for T {}

// Without `diagnostic_namespace`, the trait bound is not useful, so we add
// an implementation for `false`` to avoid a useless diagnostic.
#[cfg(not(diagnostic_namespace))]
impl<T> PyClassSync<false> for T {}

mod tests {
#[cfg(feature = "macros")]
#[test]
fn test_assert_pyclass_sync() {
use super::assert_pyclass_sync;

#[crate::pyclass(crate = "crate")]
struct MyClass {}
assert_pyclass_sync::<MyClass, true>();
}
}
72 changes: 72 additions & 0 deletions src/impl_/pyclass/probes.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
use std::marker::PhantomData;

use crate::{conversion::IntoPyObject, Py};
#[allow(deprecated)]
use crate::{IntoPy, ToPyObject};

/// Trait used to combine with zero-sized types to calculate at compile time
/// some property of a type.
///
/// The trick uses the fact that an associated constant has higher priority
/// than a trait constant, so we can use the trait to define the false case.
///
/// The true case is defined in the zero-sized type's impl block, which is
/// gated on some property like trait bound or only being implemented
/// for fixed concrete types.
pub trait Probe {
const VALUE: bool = false;
}

macro_rules! probe {
($name:ident) => {
pub struct $name<T>(PhantomData<T>);
impl<T> Probe for $name<T> {}
};
}

probe!(IsPyT);

impl<T> IsPyT<Py<T>> {
pub const VALUE: bool = true;
}

probe!(IsToPyObject);

#[allow(deprecated)]
impl<T: ToPyObject> IsToPyObject<T> {
pub const VALUE: bool = true;
}

probe!(IsIntoPy);

#[allow(deprecated)]
impl<T: IntoPy<crate::PyObject>> IsIntoPy<T> {
pub const VALUE: bool = true;
}

probe!(IsIntoPyObjectRef);

// Possible clippy beta regression,
// see https://github.com/rust-lang/rust-clippy/issues/13578
#[allow(clippy::extra_unused_lifetimes)]
impl<'a, 'py, T: 'a> IsIntoPyObjectRef<T>
where
&'a T: IntoPyObject<'py>,
{
pub const VALUE: bool = true;
}

probe!(IsIntoPyObject);

impl<'py, T> IsIntoPyObject<T>
where
T: IntoPyObject<'py>,
{
pub const VALUE: bool = true;
}

probe!(IsSync);

impl<T: Sync> IsSync<T> {
pub const VALUE: bool = true;
}
Loading
Loading