Validate literals at a later stage #12449
Labels
enhancement
Solving this issue will likely involve adding new logic or components to the codebase.
error message
This issue points out an error message that is unhelpful and should be improved.
frontend
Tokenization, parsing, AstGen, Sema, and Liveness.
Milestone
Comments
Vexu
added
enhancement
|
I'm keen to take a look at this once #12661 has been merged, but currently blocked on waiting for that or otherwise the work might potentially conflict there. |
tau-dev
added a commit
to tau-dev/zig
that referenced
this issue
Jul 12, 2024
Closes ziglang#12449 and ziglang#13809. Generate .invalid tokens only in severe cases (illegal line break or null). This allows us to continue parsing a lot more often, allowing for more and better error messages. The numeric literals mentioned in ziglang#12449 already had this treatment, this commit applies it to char literals and identifiers. In error messages, count Unicode codepoints to line up the source highlight. Render tabs as four spaces, Zig's default indentation.
tau-dev
added a commit
to tau-dev/zig
that referenced
this issue
Jul 12, 2024
Closes ziglang#12449 and ziglang#13809. Generate .invalid tokens only in severe cases (illegal line break or null). This allows us to continue parsing a lot more often, allowing for more and better error messages. The numeric literals mentioned in ziglang#12449 already had this treatment, this commit applies it to char literals and identifiers. In error messages, count Unicode codepoints to line up the source highlight. Render tabs as four spaces, Zig's default indentation.
tau-dev
added a commit
to tau-dev/zig
that referenced
this issue
Jul 12, 2024
Closes ziglang#12449 and ziglang#13809. Generate .invalid tokens only in severe cases (illegal line break or null). This allows us to continue parsing a lot more often, allowing for more and better error messages. The numeric literals mentioned in ziglang#12449 already had this treatment, this commit applies it to char literals and identifiers. In error messages, count Unicode codepoints to line up the source highlight. Render tabs as four spaces, Zig's default indentation.
tau-dev
added a commit
to tau-dev/zig
that referenced
this issue
Jul 13, 2024
Closes ziglang#12449 and ziglang#13809. Generate .invalid tokens only in severe cases (illegal line break or null). This allows us to continue parsing a lot more often, allowing for more and better error messages. The numeric literals mentioned in ziglang#12449 already had this treatment, this commit applies it to char literals and identifiers. In error messages, count Unicode codepoints to line up the source highlight. Render tabs as four spaces, Zig's default indentation.
tau-dev
added a commit
to tau-dev/zig
that referenced
this issue
Jul 14, 2024
Closes ziglang#12449 and ziglang#13809. Generate .invalid tokens only in severe cases (illegal line break or null). This allows us to continue parsing a lot more often, allowing for more and better error messages. The numeric literals mentioned in ziglang#12449 already had this treatment, this commit applies it to char literals and identifiers. In error messages, count Unicode codepoints to line up the source highlight. Render tabs as four spaces, Zig's default indentation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Most people would recognize
0x213gas a malformed hex literal and the same goes for''being an empty char literal but currently Zig validates these kinds of error in the tokenizer, giving out terrible, vague errors likeexpected expression, found 'invalid bytes'and preventing formatting.It would instead be better to have the tokenizer only validate things that actually prevent the tokens from being valid such as newlines in string literals (which should also give a proper error message instead of the same
invalid bytesone) and leave everything else to AstGen.The text was updated successfully, but these errors were encountered: