Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(semantic)!: ensure program outlives semantic #8455

Merged
merged 7 commits into from
Jan 16, 2025

Conversation

valeneiko
Copy link
Contributor

@valeneiko valeneiko commented Jan 12, 2025

fixes: #8437

In semantic builder make sure Program reference has a lifetime of the Arena.

Copy link

graphite-app bot commented Jan 12, 2025

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

@github-actions github-actions bot added A-linter Area - Linter A-parser Area - Parser A-semantic Area - Semantic A-minifier Area - Minifier A-transformer Area - Transformer / Transpiler A-codegen Area - Code Generation A-prettier Area - Prettier A-isolated-declarations Isolated Declarations A-editor Area - Editor and Language Server labels Jan 12, 2025
@valeneiko valeneiko changed the title semantic: ensure program outlives semantic fix(semantic): ensure program outlives semantic Jan 12, 2025
@github-actions github-actions bot added the C-bug Category - Bug label Jan 12, 2025
Copy link

codspeed-hq bot commented Jan 12, 2025

CodSpeed Performance Report

Merging #8455 will not alter performance

Comparing valeneiko:fix-semantic-lifetimes (d0358eb) with main (04bc259)

Summary

✅ 32 untouched benchmarks

@Boshen
Copy link
Member

Boshen commented Jan 13, 2025

pub program: &'a mut Program<'a> from ParserReturn doesn't feel right, let's see if @overlookmotel knows what the underly problem is.

@overlookmotel
Copy link
Contributor

I've written up my thoughts on this problem in #8437 (comment) and #8461.

Personally, I quite like the idea of parser returning &mut Program. BUT that's a large breaking change, and I don't think it's necessary to fix this issue. I think just changing SemanticBuilder::build to take a &'a Program<'a> should be sufficient to squash the bug.

So I suggest we go one step at a time - fix #8437 first, then discuss API changes after. I have some other thoughts about API, and if we're going to make breaking changes, it'd be preferable to do that only once.

Or have I misunderstood? Is parser returning &mut Program necessary to fix the bug?

@valeneiko
Copy link
Contributor Author

Changing parser return type is not necessary. But we would still need to update all usages of semantic to first copy the program into the allocator and then use the new reference.

let program = alloc.alloc(parserReturn.program);
let semanticReturn = SemanticBuilder::new().build(program);

@valeneiko
Copy link
Contributor Author

@overlookmotel I've updated the PR to just fix semantic.

@overlookmotel overlookmotel force-pushed the fix-semantic-lifetimes branch from 1684b92 to d0358eb Compare January 15, 2025 18:41
@overlookmotel overlookmotel changed the title fix(semantic): ensure program outlives semantic fix(semantic)!: ensure program outlives semantic Jan 15, 2025
Copy link
Contributor

@overlookmotel overlookmotel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks very much for doing this, and thanks for reverting the changes to ParserReturn.

Unfortunately this does make the API harder to use, but that's necessary to make it correct. SemanticBuilderReturn does borrow the Program for the duration of its existence, so we need to make sure it does actually work that way.

I've pushed 1 extra commit to remove unreachable_unchecked. The purpose of that part of the coverage runner is to make sure that Semantic contains what it should do, so it's best not to make unsafe assertions about the very thing we're trying to check.

Apart from that, great!

This is a breaking change, so before we merge this we should make sure it doesn't break downstream consumers in a terrible way. I'll try it out with Rolldown now, and will report back.

Copy link
Contributor

@overlookmotel overlookmotel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not causing major problems in Rolldown. Given that's a large codebase which makes heavy use of Oxc, likely it's not going to cause too many problems in other downstream projects either.

@Boshen Boshen merged commit 4ce6329 into oxc-project:main Jan 16, 2025
28 checks passed
@valeneiko valeneiko deleted the fix-semantic-lifetimes branch January 16, 2025 08:37
Boshen added a commit that referenced this pull request Jan 18, 2025
## [0.47.0] - 2025-01-18

- fae4cd2 allocator: [**BREAKING**] Remove `Vec::into_string` (#8571)
(overlookmotel)

- 95bc0d7 allocator: [**BREAKING**] `Allocator` do not deref to
`bumpalo::Bump` (#8569) (overlookmotel)

- 19d3677 ast: [**BREAKING**] Always return
`Array<ImportDeclarationSpecifier>` for `ImportDeclaration.specifiers`
(#8560) (sapphi-red)

- 4ce6329 semantic: [**BREAKING**] Ensure program outlives semantic
(#8455) (Valentinas Janeiko)

- 7066d1c ast, span, syntax, regular_expression: [**BREAKING**] Remove
`ContentHash` (#8512) (overlookmotel)

### Features

- bf4e5e1 allocator: Add `HashMap` (#8553) (overlookmotel)
- a6d71f8 ast: Add `AstKind::ty` method (#8521) (overlookmotel)
- 4d4e805 minifier: Collapse if stmt with empty consequent (#8577)
(camc314)
- 991a22f minifier: Fold `Array::concat` into literal (#8442)
(sapphi-red)
- 3dc2d8b minifier: Fold string concat chaining (#8441) (sapphi-red)
- a4ae450 minifier: Fold array concat chaining (#8440) (sapphi-red)
- 7cc81ef minifier: Fold invalid typeof comparisons (#8550) (camc314)
- 927f43f minifier: Improve `.charCodeAt(arg)` when arg is valid (#8534)
(Boshen)
- 06f14d5 minifier: Remove empty class static block `class Foo { static
{} }` (#8525) (Boshen)
- 1860411 minifier: Remove last redundant return statement (#8523)
(Boshen)
- c479a58 napi/parser: Expose dynamic import expressions (#8540)
(Boshen)
- 2f0314e npm/oxc-minify: Npm package and publish script (#8579)
(Boshen)
- f413bb5 transformer/optional-chaining: Change parent scope for
expression when it wrapped with an arrow function (#8511) (Dunqing)

### Bug Fixes

- e87c001 allocator: Statically prevent memory leaks in allocator
(#8570) (overlookmotel)
- 855c839 codegen: Shorthand assignment target identifier consider
mangled names (#8536) (Boshen)
- 65c596d minifer: Keep idents if not in scope when minimizing array
exprs (#8551) (camc314)
- f57aac2 minifier: Incorrect folding of expr in bool ctx (#8542)
(camc314)
- 946ad76 minifier: `(-Infinity).toString()` -> `'-Infinity'` (#8535)
(Boshen)
- b1d0186 minifier: Do not fold `!!void b` (#8533) (Boshen)
- 53adde5 minifier: `x['-2147483648']` -> `x[-2147483648]` (#8528)
(Boshen)
- 405b73d minifier: Do not change `delete undefined` to `delete void 0`
(#8527) (Boshen)
- 92e44cb minifier: Do not remove `undefined` in `var x = undefined`
(#8526) (Boshen)
- 209e313 minifier: `class C { ['-1']() {} }` cannot be minifized
(#8516) (Boshen)
- 6585463 minifier: Always keep the last value of sequence expression
(#8490) (Boshen)
- b552f5c transformer: `wrap_in_arrow_function_iife` take span of input
`Expression` (#8547) (overlookmotel)
- 9963533 transformer/arrow-functions: Visit arguments to `super()` call
(#8494) (overlookmotel)
- 06ccb51 transformer/async-to-generator: Move parameters to the inner
generator function when they could throw errors (#8500) (Dunqing)
- 356f0c1 transformer/class-properties: Handle nested `super()` calls
(#8506) (overlookmotel)
- a048337 transformer/class-static-blocks: Static block converted to
IIFE use span of original block (#8549) (overlookmotel)

### Performance

- 76ea52b allocator: Inline `Box` methods (#8572) (overlookmotel)
- 93df57f allocator: `#[inline(always)]` methods of `Vec` which just
delegate to `allocator_api2` (#8567) (overlookmotel)
- 5a28d68 allocator: `#[inline(always)]` methods of `HashMap` which just
delegate to `hashbrown` (#8565) (overlookmotel)
- d17021c mangler: Optimize `base54` function (#8557) (overlookmotel)
- 6b52d7a mangler: Use a single allocation space for temporary vecs
(#8495) (Boshen)
- 30a869e semantic: Use `oxc_allocator::HashMap` in `ScopeTree` (#8554)
(overlookmotel)
- 63eb298 span: Compare `Span`s as single `u64`s (#8300) (overlookmotel)
- a43560c span: Hash `Span` as a single `u64` (#8299) (overlookmotel)
- 3fff7d2 span: Align `Span` same as `usize` (#8298) (overlookmotel)
- 53ef263 transformer/arrow-functions: Bail out of visiting early when
inserting `_this = this` after `super()` (#8482) (overlookmotel)

### Documentation

- fa1a6d5 allocator: Update docs for `Vec` (#8555) (overlookmotel)

### Refactor

- ac05134 allocator: `String` type (#8568) (overlookmotel)
- 68fab81 allocator: Rename inner `Vec` type (#8566) (overlookmotel)
- fcbca32 ast: Rename `#[estree(with)]` to `#[estree(via)]` (#8564)
(overlookmotel)
- 007e8c0 ast, regular_expression: Shorten `ContentEq` implementations
(#8519) (overlookmotel)
- b4c87e2 linter: Move DiagnosticsReporters to oxlint (#8454) (Alexander
S.)
- 8f57929 minifier: Merge `try_compress_type_of_equal_string` into
`try_minimize_binary` (#8561) (sapphi-red)
- 2857ae1 parser: Refactor visitor in regexp example (#8524)
(overlookmotel)
- b5ed58e span: All methods take owned `Span` (#8297) (overlookmotel)
- 712633f transformer: `wrap_statements_in_arrow_function_iife` utility
function (#8548) (overlookmotel)
- 5206c6a transformer: Rename `wrap_in_arrow_function_iife` (#8546)
(overlookmotel)
- 61077ca transformer: `wrap_arrow_function_iife` receive an owned
`Expression` (#8545) (overlookmotel)
- 6820d24 transformer: Move `wrap_arrow_function_iife` to root utils
module (#8529) (Dunqing)
- 52bd0b1 transformer: Move common utils functions to the root module
(#8513) (Dunqing)
- c30654a transformer/arrow-function: Wrapping arrow function iife by
using `wrap_arrow_function_iife` (#8530) (Dunqing)
- 2bc5175 transformer/arrow-functions: Rename method (#8481)
(overlookmotel)
- 72f425f transformer/class-properties: Fix lint warning in release mode
(#8539) (overlookmotel)
- 7e61b23 transformer/typescript: Shorten code (#8504) (overlookmotel)
- 04bc259 traverse: Remove unnecessary `#[allow]` (#8518)
(overlookmotel)
- a368726 traverse: Harden soundness of `Traverse` and document safety
invariants better (#8507) (overlookmotel)

### Testing

- e0f5d6c minifier: Update esbuild test (Boshen)
- 629c417 minifier: Port esbuild minification tests (#8497) (Boshen)

Co-authored-by: Boshen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-codegen Area - Code Generation A-editor Area - Editor and Language Server A-isolated-declarations Isolated Declarations A-linter Area - Linter A-minifier Area - Minifier A-parser Area - Parser A-prettier Area - Prettier A-semantic Area - Semantic A-transformer Area - Transformer / Transpiler C-bug Category - Bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ast: Out of bounds memory access crash in AST Vititor
3 participants