Clarification on the variance of self on Protocols

[MathiasSven]

I am not sure pyright is the right place to open this as a bug report as I am not sure what the consensus is on the expected behavior, regardless to me it seems that when annotating a Protocol this construction should be allowed:

class A(Protocol[_T_co]):
     def foo(self: _T_co): ...

Currently mypy accepts this, however pyright complains that since the type variable is used as a parameter it ought to be contravariant, conversely if you set the type variable to contravariant, then mypy complains about the opposite. Due to the nature of self should it not be treated as covariant?

As to how this could be useful, I think broadly not so much, but it does is give you access to the type when annotating functions with such a protocol. In my specific use case:

class A(Protocol[_T_co, _T_contra, _R_co]):
     def foo(self: _T_co, __other: _T_contra) -> _R_co: ...

def bar(x: A[_T, _T1, _T2], y: A[_T1, _T, _T3]) -> _T2 | _T3:
    a = x.foo(y) # Safe, will return _T2
    b = y.foo(x) # Safe, will return _T3
    
    if some_condition:
        return a
    else:
        return b

[erictraut]

Converting to discussion.

[MathiasSven]

After looking a little further into it, even using mypy where I am allowed to type the self parameter as covariant, that does not really solve my issue, I think I just don't fully understand how typevars resolve in this case:

Upon further inspection, the reason that mypy did not solve my issue appears to be a bug with mypy:

class ProtoFoo(Protocol[_T_co]):
    def foo(self: _T_co) -> None: ...

class Foo:
    def foo(self) -> None: ...

class SubFoo(Foo): ...
    
a: ProtoFoo[SubFoo] = Foo()  # Fine, but shouldn't

Since mypy complains if self is typed anything other then covariant, I believe this is intended to be supported, so I might raise it as an issue on mypy. However, this kind of construction can't be used without consensus on the correct behavior among type checkers so I would still like to know your thoughts on this before raising the issue.

[erictraut]

Independent of the question about whether the type of the self parameter should be enforced as covariant or otherwise, I don't think this pattern is valid. The problem is that you are creating a protocol where the implied type of self is the protocol class itself, but then you're making the protocol class generic and parameterizing it with a different type that is then applied to the self parameter. The correct type of self in this case is ProtoFoo. Annotating it as _T_co is somewhat nonsensical.

[erictraut]

Within non-protocol classes, pyright and mypy both enforce that a type annotation on self is a supertype of the containing class.

For example:

class Parent:
    ...

class Child(Parent):
    def func1(self: Parent) -> None: # Allowed
        ...
    def func2(self: int) -> None:  # Error
        ...
    def func3(self: "Grandchild") -> None:  # Error
        ...

class Grandchild(Child):
    ...

This is consistent with the idea that the type of self is contravariant rather than covariant.

Neither pyright nor mypy enforce this "self type must be a supertype of the containing class" rule for protocol classes. That's because the notion of a "subtype" is more complex with structural subtyping, and recursion can easily result when determining whether one protocol is a subtype of another. But perhaps we should attempt to enforce this rule for protocol classes. If it were enforced, then _T_co would clearly be an illegal annotation for self because it is not a supertype of the containing class.

[MathiasSven]

The problem is that you are creating a protocol where the implied type of self is the protocol class itself, but then you're making the protocol class generic and parameterizing it with a different type that is then applied to the self parameter. The correct type of self in this case is ProtoFoo. Annotating it as _T_co is somewhat nonsensical.

I do not see how this is nonsensical, a protocol is just a structural contract, if I have:

class Foo: ...

class ProtoFoo(Protocol):
    def foo(self: Foo) -> Any: ...

ProtoFoo is a valid structural contract, where any type is a subtype of it iff it implements a foo method which takes Foo as a self parameter, the fact that Foo is actually not compatible with ProtoFoo should not be a concern of the protocol construction, but rather of the protocol assignment. A protocol never gets instantiated, it's methods are never bounded, there is no obligation that self is of type ProtoFoo, the only implicit obligation is that self be a subtype of ProtoFoo. The construction I am suggesting does not change the semantics of the behavior of a protocol nor does it seem make anything unsound (at least as I see it right now), it only adds the ability to further specify its subtype properties, eg.: "that it be a subtype of Foo as well".

This is consistent with the idea that the type of self is contravariant rather than covariant.

I do not see how that example relates to covariance or contravariance, would that not just be a consequence of the fact that when you create an element of the type Child it will bind those methods with Child as the type for the self parameter, and Child is not compatible with Grandchild nor int? On the other hand, this shows the opposite, that "the first parameter on methods" is covariant:

class A: ...
class SubA(A): ...
 
def foo(self: SubA, x: int) -> Any: ...

a: Callable[[A, int], Any] = foo  # Error as Callable[[SubA, int], Any] is not a subtype of Callable[[A, int], Any]

class Parent:
    def func1(self: 'Parent', x: int) -> Any: ...
    def func2(self: 'Parent', x: int) -> Any: ...

class Child(Parent):
    def func1(self: 'Child', x: int) -> Any: ...  # This is Allowed, it does not make anything unsound, 
                                                  # however were it not a method, this assignemnt would not be allowed.

    def func2(self: 'Child', x: bool) -> Any: ...  # This is an error due to contravaraince, but it only applies 
                                                   # to the parameters that are not bounded

[MathiasSven]

Regardless of this, I think whatever a typed self parameter on a Protocol would achieve, would also be achievable by a generic intersection of a protocol and a nominal class, when or if that ever becomes a thing.