Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(next-custom-transforms): Replace non-Sync Rc<DashMap<_, _>> usage with Rc<RefCell<FxHashMap<_, _>>> #75534

Merged
merged 1 commit into from
Feb 4, 2025

Conversation

bgw
Copy link
Member

@bgw bgw commented Jan 31, 2025

This is a follow-up to #75007

DashMap is needed for situations where you have many threads (or tokio tasks) modifying the same data at the same time. It introduces a lot of memory and CPU overhead to create individually lockable RwLock shards. This type of data structure implements Sync, indicating that it can be shared between threads.

Rc is not Sync (Arc is Sync), so wrapping DashMap in Rc doesn't make a lot of sense: You're paying a pretty high overhead for the ability to share data between threads, and wrapping it in something that does not support sharing data between threads.

This visitor code is entirely single-threaded, so just use Rc<RefCell<FxHashMap>> instead. RefCell is a very cheap single-threaded version of a Mutex. FxHashMap is just a HashMap with the faster FxHash algorithm used.

Closes PACK-3871

@bgw bgw requested review from gaojude and ijjk January 31, 2025 20:07
@bgw bgw marked this pull request as ready for review January 31, 2025 20:12
@bgw bgw changed the title perf(next-custom-transforms): Replace non-Sync Rc<DashMap<_, _>> usage with Rc<RefCell<FxHashMap<_, _>>> perf(next-custom-transforms): Replace non-Sync Rc<DashMap<_, _>> usage with Rc<RefCell<FxHashMap<_, _>>> Jan 31, 2025
@ijjk
Copy link
Member

ijjk commented Feb 3, 2025

Stats from current PR

Default Build (Increase detected ⚠️)
General
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
buildDuration 19.2s 17.7s N/A
buildDurationCached 16.9s 14.3s N/A
nodeModulesSize 391 MB 391 MB
nextStartRea..uration (ms) 467ms 452ms N/A
Client Bundles (main, webpack)
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
5306-HASH.js gzip 54 kB 54 kB N/A
8276.HASH.js gzip 169 B 168 B N/A
8377-HASH.js gzip 5.46 kB 5.46 kB N/A
bccd1874-HASH.js gzip 52.9 kB 52.9 kB N/A
framework-HASH.js gzip 57.5 kB 57.5 kB N/A
main-app-HASH.js gzip 239 B 241 B N/A
main-HASH.js gzip 34.5 kB 34.4 kB N/A
webpack-HASH.js gzip 1.71 kB 1.71 kB N/A
Overall change 0 B 0 B
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Overall change 39.4 kB 39.4 kB
Client Pages
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
_app-HASH.js gzip 193 B 193 B
_error-HASH.js gzip 193 B 193 B
amp-HASH.js gzip 512 B 510 B N/A
css-HASH.js gzip 343 B 342 B N/A
dynamic-HASH.js gzip 1.84 kB 1.84 kB
edge-ssr-HASH.js gzip 265 B 265 B
head-HASH.js gzip 363 B 362 B N/A
hooks-HASH.js gzip 393 B 392 B N/A
image-HASH.js gzip 4.59 kB 4.58 kB N/A
index-HASH.js gzip 268 B 268 B
link-HASH.js gzip 2.35 kB 2.35 kB N/A
routerDirect..HASH.js gzip 328 B 328 B
script-HASH.js gzip 397 B 397 B
withRouter-HASH.js gzip 323 B 326 B N/A
1afbb74e6ecf..834.css gzip 106 B 106 B
Overall change 3.59 kB 3.59 kB
Client Build Manifests
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
_buildManifest.js gzip 748 B 747 B N/A
Overall change 0 B 0 B
Rendered Page Sizes
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
index.html gzip 524 B 524 B
link.html gzip 539 B 538 B N/A
withRouter.html gzip 520 B 520 B
Overall change 1.04 kB 1.04 kB
Edge SSR bundle Size
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
edge-ssr.js gzip 129 kB 129 kB N/A
page.js gzip 210 kB 210 kB N/A
Overall change 0 B 0 B
Middleware size
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
middleware-b..fest.js gzip 670 B 663 B N/A
middleware-r..fest.js gzip 155 B 156 B N/A
middleware.js gzip 31.3 kB 31.3 kB N/A
edge-runtime..pack.js gzip 844 B 844 B
Overall change 844 B 844 B
Next Runtimes
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
app-page-exp...dev.js gzip 385 kB 385 kB
app-page-exp..prod.js gzip 132 kB 132 kB
app-page-tur..prod.js gzip 145 kB 145 kB
app-page-tur..prod.js gzip 141 kB 141 kB
app-page.run...dev.js gzip 372 kB 372 kB
app-page.run..prod.js gzip 128 kB 128 kB
app-route-ex...dev.js gzip 39.3 kB 39.3 kB
app-route-ex..prod.js gzip 25.6 kB 25.6 kB
app-route-tu..prod.js gzip 25.6 kB 25.6 kB
app-route-tu..prod.js gzip 25.4 kB 25.4 kB
app-route.ru...dev.js gzip 40.8 kB 40.8 kB
app-route.ru..prod.js gzip 25.4 kB 25.4 kB
dist_client_...dev.js gzip 356 B 356 B
dist_client_...dev.js gzip 349 B 349 B
pages-api-tu..prod.js gzip 9.69 kB 9.69 kB
pages-api.ru...dev.js gzip 11.8 kB 11.8 kB
pages-api.ru..prod.js gzip 9.68 kB 9.68 kB
pages-turbo...prod.js gzip 21.9 kB 21.9 kB
pages.runtim...dev.js gzip 31.5 kB 31.5 kB
pages.runtim..prod.js gzip 21.9 kB 21.9 kB
server.runti..prod.js gzip 60.2 kB 60.2 kB
Overall change 1.65 MB 1.65 MB
build cache Overall increase ⚠️
vercel/next.js canary vercel/next.js bgw/use-cache-fxhashmap Change
0.pack gzip 2.1 MB 2.1 MB ⚠️ +490 B
index.pack gzip 74 kB 74.6 kB ⚠️ +541 B
Overall change 2.18 MB 2.18 MB ⚠️ +1.03 kB
Diff details
Diff for main-HASH.js

Diff too large to display

Commit: 967c0cd

@bgw bgw force-pushed the bgw/update-rustc-hash branch from 83ff8d9 to 2e57d75 Compare February 3, 2025 21:55
@bgw bgw force-pushed the bgw/use-cache-fxhashmap branch from 2aa5104 to 6e80bfb Compare February 3, 2025 21:55
@bgw bgw force-pushed the bgw/update-rustc-hash branch 2 times, most recently from 0edd4df to 4711122 Compare February 3, 2025 23:00
@bgw bgw force-pushed the bgw/use-cache-fxhashmap branch from 6e80bfb to 9865cba Compare February 3, 2025 23:01
@bgw bgw force-pushed the bgw/update-rustc-hash branch from 4711122 to c4a1fce Compare February 4, 2025 01:41
@bgw bgw force-pushed the bgw/use-cache-fxhashmap branch from 9865cba to 60214f7 Compare February 4, 2025 01:41
@bgw bgw changed the base branch from bgw/update-rustc-hash to graphite-base/75534 February 4, 2025 02:59
@bgw bgw force-pushed the graphite-base/75534 branch from c4a1fce to fd27e58 Compare February 4, 2025 03:00
@bgw bgw force-pushed the bgw/use-cache-fxhashmap branch from 60214f7 to 18621bc Compare February 4, 2025 03:00
@bgw bgw changed the base branch from graphite-base/75534 to canary February 4, 2025 03:01
@bgw bgw force-pushed the bgw/use-cache-fxhashmap branch from 18621bc to 967c0cd Compare February 4, 2025 03:01
@bgw bgw merged commit 7dcc35c into canary Feb 4, 2025
131 checks passed
Copy link
Member Author

bgw commented Feb 4, 2025

Merge activity

  • Feb 3, 10:23 PM EST: A user merged this pull request with Graphite.

@bgw bgw deleted the bgw/use-cache-fxhashmap branch February 4, 2025 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
created-by: Turbopack team PRs by the Turbopack team. Turbopack Related to Turbopack with Next.js.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants