[flake8-annotations] Fix parsing of "complex" forward reference type annotations in any-type (ANN401)
#14700
Conversation
|
|
Do you think this is an issue that should be solved holistically (#10586)? Because the |
Sorry, I'm a bit confused still. Why are we using the range without quotes for the parsed expression in case of complex annotation? Because for simple annotations, we just use the range without quotes for the original string expression. |
|
I think an ideal solution here would be to fix the |
| relocate_expr(parsed.expr_mut(), string_expr.range()); | ||
| // Remove quotes from nested forward reference type annotations | ||
| let range_excluding_quotes = | ||
| if let Expr::StringLiteral(ExprStringLiteral { ref value, .. }) = *parsed.syntax.body { | ||
| let string_parts = value.as_slice(); | ||
| let start = string_parts[0].flags.opener_len(); | ||
| let end = string_parts[string_parts.len() - 1].flags.closer_len(); | ||
| string_expr.range().add_start(start).sub_end(end) | ||
| } else { | ||
| string_expr.range() | ||
| }; | ||
| relocate_expr(parsed.expr_mut(), range_excluding_quotes); |
There was a problem hiding this comment.
The way it would work with simple type annotations is that it first parses the outer string and then the inner string on the next call:
parse_simple_type_annotation: "'int'" (10..15)
parse_simple_type_annotation: "int" (11..14)
In the case of complex annotations, it seems like the range is always going to be the one without the inner quotes:
parse_complex_type_annotation: "'int'" (10..18)
parse_complex_type_annotation: "int" (10..18)
Summary
Fixes #14695.
The root cause was tricky to find.
"Complex" forward reference type annotations are handled differently. Forward references are "complex" in this case when the raw contents (without quotes) of the expression with the parsed contents is not contained in the string literal.
ruff/crates/ruff_python_parser/src/typing.rs
Line 65 in 84748be
Annotations are recursively parsed, for instance:
"'int'"(complex) =>"int"(simple)The raw contents of the expression "in\x74" is not contained in the string literal
intand hence requires multiple complex parsing steps:"'in\x74'"(complex) ->"in\x74"(complex). This is rare, and probably that's why the bug went unnoticed.Parsed type annotations are cached by their start offset:
ruff/crates/ruff_linter/src/checkers/ast/mod.rs
Line 2544 in 84748be
As it turned out, for these complex annotations the starting offset was not properly updated (removing the quotes), as well already do for the simple annotations. This caused an infinite loop, and eventually a stack overflow.
Test Plan
Extended the test suite with regression tests.