WIP: Add type annotations to base spec #1676
Conversation
edgarrmondragon
changed the title
edgarrmondragon
force-pushed
the
types-abc-impl
branch
from
9857dbd to
199b92e
Compare
|
Thanks for giving this a start. Perhaps you already appreciate that this process will not be easy for the package, and spec might have been the hardest place to start (since all these methods are overridden in various duck ways in the backend implementations). |
|
Preface: Just a random user. This is a really good start (I love me some types), though I feel like it could do with some minor improvements
...however, I feel like it could just devolve into a bunch of minor nitpicks, whereas this already provides lots of value as-is. What's the strategy towards a merge here? Grab a bunch of popular downstream consumers and ensure these don't cause conflicts with any overloaded duck typing happening there? |
|
@mon : as with my initial comments, I am happy to see typing leak slowly into fsspec, but the duck-type design of the (old!) code means that it will be hard to have it everywhere and certainly very hard to get mypy happy throughout the codebase, including downstream implementations. It gets complicated fast, because of issues like people demanding support for Path instances where originally only strings were allowed. The end-user API is based on convenience, but library developers using fsspec want strict definitions - requirements that are hard to simultaneously meet.
It shouldn't be possible to break code with types alone, right? |
At runtime, there will be no effect, unless you type annotate forward/circular references. But these are found as soon as the module is imported, so shouldn't ever be a problem for you, only the person adding the types (and easily fixed by gating the import behind an My worry was there might just be some downstream projects that, for example, extend the valid argument types for some functions, and making them specifically typed (instead of the default |
| @@ -381,7 +405,7 @@ def _ls_from_cache(self, path): | |||
| except KeyError: | |||
| pass | |||
|
|
|||
| def walk(self, path, maxdepth=None, topdown=True, on_error="omit", **kwargs): | |||
| def walk(self, path: str, maxdepth=None, topdown=True, on_error="omit", **kwargs): | |||
There was a problem hiding this comment.
nit: on_error has only a few valid values, right? We should make it a Union of Type Literals (should probably consider setting that as a TypeVar).
| return f"<File-like object {type(self.fs).__name__}, {self.path}>" | ||
|
|
||
| __repr__ = __str__ | ||
|
|
||
| def __enter__(self): | ||
| def __enter__(self) -> "AbstractBufferedFile": |
There was a problem hiding this comment.
May want to consider typing_extensions.Self, if it's not already a project dependency, we should add as it's one of the top 10 most popular pypi downloads anyway.
|
Nit: we may want to enable the relevent type checking rules in RUFF at some point. |
| @@ -606,7 +637,7 @@ def glob(self, path, maxdepth=None, **kwargs): | |||
| depth_double_stars = path[idx_double_stars:].count("/") + 1 | |||
| depth = depth - depth_double_stars + maxdepth | |||
| else: | |||
| depth = None | |||
| depth = None # type: ignore[assignment] | |||
There was a problem hiding this comment.
Type depth above as an Optional and you won't need this type ignore.
In Python 3.9 or newer, it should not. |
|
@martindurant I have a lot of typing experience and am happy to look at this PR if you would like. |
|
@Skylion007 yes please, much appreciated! |
See #625 (comment)