Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor[parser]: remove ASTTokens #4364

Open
wants to merge 39 commits into
base: master
Choose a base branch
from

Conversation

charles-cooper
Copy link
Member

@charles-cooper charles-cooper commented Nov 18, 2024

What I did

remove asttokens from the parse machinery, since the method is buggy (see below bugs) and slow (#4272). brings down parse time (time spent in ast generation) between 40-70%.

fixes multiple tokenizer-related bugs, including

How I did it

another way to adjust tokens

How to verify it

Commit message

Commit message for the final, squashed PR. (Optional, but reviewers will appreciate it! Please see our commit message style guide for what we would ideally like to see in a commit message.)

Description for the changelog

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

vyper/ast/parse.py Fixed Show fixed Hide fixed
@charles-cooper charles-cooper changed the title refactor[lang]: remove ASTTokens refactor[parser]: remove ASTTokens Jan 1, 2025
Copy link

codecov bot commented Jan 1, 2025

Codecov Report

Attention: Patch coverage is 89.02439% with 9 lines in your changes missing coverage. Please review.

Project coverage is 45.01%. Comparing base (4507d2a) to head (042021c).

Files with missing lines Patch % Lines
vyper/ast/parse.py 91.52% 4 Missing and 1 partial ⚠️
vyper/ast/pre_parser.py 82.60% 4 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (4507d2a) and HEAD (042021c). Click for more details.

HEAD has 179 uploads less than BASE
Flag BASE (4507d2a) HEAD (042021c)
180 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4364       +/-   ##
===========================================
- Coverage   91.94%   45.01%   -46.93%     
===========================================
  Files         119      119               
  Lines       16677    16716       +39     
  Branches     2807     2810        +3     
===========================================
- Hits        15333     7525     -7808     
- Misses        924     8627     +7703     
- Partials      420      564      +144     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@charles-cooper charles-cooper marked this pull request as ready for review January 1, 2025 01:58
easier to translate to milliseconds
@charles-cooper charles-cooper added this to the v0.4.1 milestone Jan 2, 2025
@charles-cooper charles-cooper added the release - must release blocker label Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release - must release blocker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants