Reduce freezes due to RPC calls

[raisjn]

Running into slow Aerospace?

EDIT: 12/05: ported perf patch to latest origin/main branch

Try out this branch.

Usage

Recompile the debug or release build and add the following lines to your aerospace.toml:

new-window-detection-timeout = 100

@liamaharon suggests the following may have better results:

new-window-detection-timeout = 50
new-window-detection-debounce = 100

What it does: new-window-detection-timeout is how long to block the main thread (up to X ms), while new-window-detection-debounce will coalesce calls within X ms together.

How it works

The root cause of slowness is that Aerospace iterates over windows and uses accessibility APIs to communicate with each app. If any application is slow to respond, it blocks Aerospace's main thread from executing. To prevent Aerospace from hanging, we put the communication with apps into a new thread.

Details

detectNewWindowsAndAttachThemToWorkspaces calls into a debouncer and a separate queue when called from refreshSession and wait for up to X ms for it to complete. validateStillPopups is behind a similar debouncer + dispatch queue.

Note: Thread safety: for detectNewWindowsAndAttachThemToWorkspace to be threadsafe, MacWindow.get() has an NSLock placed in it so that multiple threads can access it safely. Additionally, specific work within MacWindow.get() should be moved to the main thread using a DispatchQueue

Observed Bugs:

Bug A: Starting a popup window and dragging it across monitors doesn't work (for example, System Settings). The window will get stuck on a single monitor and keep hopping back. Dragging the window within a single monitor works fine. It is possible to move the window via keyboard shortcuts, though. (I think this happens on all approaches? I haven't tested stock Aerospace)

Bug B: Focus follows mouse (AutoRaise application) doesn't play well with aerospace and multiple monitors. Switching the workspace on one monitor (monitor 1), then moving to another monitor (monitor 2) with the mouse will often cause the workspace on monitor 1 to switch back to the previous workspace. (I think this happens on all approaches? I haven't tested stock Aerospace)

FAQ

1. When will this be merged upstream? Answer: I have no plan to attempt to upstream this. I keep a private fork with my own changes. I've released this particular patch because people have been asking for the last year for performance improvements.
2. How do I install this? Answer: See @wojciech-kulik's comment
3. Did Aerospace crash because of this patch? Answer: Maybe. I believe Aerospace itself is prone to crashing.

[raisjn]

Thanks to @raisjn for doing the performance experiments. The debouncing approach makes sense. The refreshSession may be called too often when a lot of things happen on the screen (e.g. when you switch the workspace), and given the idempotent nature of refreshSession, debouncing the whole session becomes trivial and may increase the app responsiveness. I don't agree with the specifics of implementation, but the general idea makes sense, thanks for your experiments. I will consider "the debouncer" approach when start working on performance.

@nikitabobko: note my comment about debouncing refreshSession having some small bugs in it. if you can get it working, that is great! overall, I don't think solely debouncing refreshSession will be enough to stop hangs: if any app misbehaves, you will eat the 1s timeout (per app) and that is already too much delay.

PS: iiuc, refreshSession is idempotent if no body is passed in, but when a body is passed in, then it is executing code that has side effects.

As correctly pointed by @raisjn, please discuss the details of @raisjn fix in #655

if you can re-parent comments from #131 here, that would be great

[nikitabobko]

Sorry for the late response, I was on a vacation

note my comment about debouncing refreshSession having some small bugs in it

If you describe what bugs you observed, maybe I can quickly tell you what was wrong. But I think you will rather have more bugs (especially concurrent bugs) if you try to debounce only the part of the refreshSession

PS: iiuc, refreshSession is idempotent if no body is passed in, but when a body is passed in, then it is executing code that has side effects.

Yes

I don't think solely debouncing refreshSession will be enough to stop hangs: if any app misbehaves, you will eat the 1s timeout (per app) and that is already too much delay.

Yes, #131 (comment)

if you can re-parent comments from #131 here, that would be great

GitHub doesn't allow it it. It's only possible to convert the whole issue to the discussion

[liamaharon]

fyi @raisjn (and anyone else who may be experiencing this issue)

new-window-detection-timeout = 50
new-window-detection-debounce = 100

on dev/okay/2024_11_02_less_cpu_lockups is working wonderfully for me in initial experiments. Much appreciated ❤️

[vianneychin]

I'd love to try your build! Stupid question, but what would the step by step look like to install the forked version? I normally get it from homebrew.

[wojciech-kulik]

The warning tells what's wrong. Try:

PATH="/path/to/repo/.deps/cargo-root/bin:$PATH" ./install-from-sources.sh

[quarkquartet]

The warning tells what's wrong. Try:

PATH="/path/to/repo/.deps/cargo-root/bin:$PATH" ./install-from-sources.sh

Thank you! That almost solved the problem. I think I got stuck now in the last step. Sorry I'm not familiar with how an Mac App is build.

[wojciech-kulik]

You should have aerospace-codesign-certificate in your Keychain. I'm not sure why it's missing on your side.

[quarkquartet]

You should have aerospace-codesign-certificate in your Keychain. I'm not sure why it's missing on your side.

Thank you for reminder! I searched and found the instruction about creating this keychain. Now I successfully installed it!

[wojciech-kulik]

Here is the instruction: https://github.com/nikitabobko/AeroSpace/blob/main/dev-docs/development.md#2-create-codesign-certificate if anyone else faces the same issue.

[PhotonQuantum]

Thank you for your excellent work! While the original version is slow to the point of being unusable, dev/okay/2024_12_05_less_cpu_lockups works like a charm on my setup! Is there any plan to get this merged upstream?

[voidpointer0]

Would love to know too, hoping this will help with the system becoming unusable after launching Unity.

[quarkquartet]

Yeah this seems to be a nice solution. Currently it's really lagging as long as I'm running some code on my laptop.

[PhotonQuantum]

Followup: so today I got a runtime error on this version
https://gist.github.com/PhotonQuantum/801a680257ceded3d3ff5fc662557fe2

I wasn't doing anything particular when it crashed. The lid was closed at that time. I have an external monitor setup that could be related to disconnection. Still, I can't be sure because I can't find any relevant event in the system log at that exact timestamp.

@raisjn I'm uncertain if your fork introduces the issue or if it is a general problem from upstream, so I'm posting it here for now.

[quarkquartet]

@raisjn Thank you for your solution. Just make sure I understand it correctly: the version installed from Homebrew does not have your implementation, and I have to install it from your forked repo with dev/okay/xxx_less_cpu_lockups right?

[grapexy]

Just wanted to report that I've been running 2024_12_05_less_cpu_lockups for about 3 weeks now without any issues. I was having issues with upstream version of aerospace locking up when running some local LLMs, and all these issues are now gone.

[vianneychin]

The performance is just really good as someone who went through the trouble of compiling a forked version. Spent literally 2 hours figuring it out but was worth it lol

[voidpointer0]

Any estimate on when/if this will be merged to main? I could really use it, but I don't want to mess with my current installation.

[wojciech-kulik]

@raisjn would it be possible to update your branch to include latest changes from the main repo?

[raisjn]

i rebased and pushed 2025_01_12_less_cpu_lockups, let me know how it works. i will try it out this week as well

is there any particular reason you wanted the latest changes from main?

[wojciech-kulik]

I just wanted to make sure that all the fixes are included, especially that I saw some crash fixes :). Thank you very much!

[wojciech-kulik]

I just got this crash :(

##### AeroSpace Runtime Error #####

Please report to:
    https://github.com/nikitabobko/AeroSpace/discussions/categories/potential-bugs
    Please describe what you did to trigger this error

Message: 
Version: 0.0.0-SNAPSHOT
Git hash: 789435f1e65c4986abbc51e90e662b252f57b94a
Date: 2025-01-13 21:57:20 +0000
Coordinate: AppBundle/MacWindow.swift:354:10 onWindowDetected(_:startup:)
recursionDetectorDuringFailure: false
cli: false
refreshSessionEvent: nil
Displays have separate spaces: true

Stacktrace:
0   AeroSpace                           0x000000010243f6f0 $s6Common6errorT_4file4line6column8functionxSS_SSS2iSStlFs5NeverO_Tg5 + 1388
1   AeroSpace                           0x000000010246b170 $s6Common5check__4file4line6column8functionySb_SSyXKSSS2iSStF + 92
2   AeroSpace                           0x0000000102304160 $s9AppBundle16onWindowDetected33_AFFA185B3FDC2EA3EF5AFA96A7308179LL_7startupyAA0D0C_SbtF + 428
3   AeroSpace                           0x00000001022f3d58 $s9AppBundle19validateStillPopups33_EA737387E8EE73FEB20FFAD7FE91D51ELL7startupySb_tF + 276
4   AeroSpace                           0x000000010231e024 $s9AppBundle15TimeoutDetectorC3run4task8onFinishyyACc_yACctFyyYbcfU_ + 44
5   AeroSpace                           0x000000010231dcf0 $sIeg_IeyB_TRTm + 28
6   libdispatch.dylib                   0x00000001808f28f8 _dispatch_call_block_and_release + 32
7   libdispatch.dylib                   0x00000001808f4658 _dispatch_client_callout + 20
8   libdispatch.dylib                   0x00000001809064e8 _dispatch_root_queue_drain + 860
9   libdispatch.dylib                   0x0000000180906b20 _dispatch_worker_thread2 + 156
10  libsystem_pthread.dylib             0x0000000180aa339c _pthread_wqthread + 228
11  libsystem_pthread.dylib             0x0000000180aa20f0 start_wqthread + 8

[raisjn]

after testing for the last week, i've found it to be unstable - likely a combination of upstream changes and not properly guarding against them. i've moved back to using the previous version

[wojciech-kulik]

I wonder if we ever get to the point where Aerospace works well without "3rd-party" hacks :(... It seems like the developer is not really interested in fixing this problem. The only reason I use Aerospace is to switch between workspaces instantly and that feature is not working well.

Maybe it would be a good idea to create another open-source project that manages virtual workspaces without any windows tiling and we could focus on making it blazingly fast instead of this laggy behavior.

[wojciech-kulik]

📢 It doesn’t seem like it’s ever going to be fixed, so I decided to start working on my own virtual workspace manager focused only on switching workspaces without tiling. After a few hours, I finished an MVP, which you can check out here:

https://github.com/wojciech-kulik/FlashSpace

• This solution seems to work fine even with high CPU usage
• It's probably buggy, I've tested it only on my own setup
• Even if my approach is not sufficient, now we've got a solid base to experiment and find the best solution. There is only one place that handles all the logic of switching workspaces.

[nikitabobko]

It doesn’t seem like it’s ever going to be fixed

It seems like the developer is not really interested in fixing this problem

@wojciech-kulik The way you talk is rude and demotivating (not necessarily towards me. For example, this is also not a good way to communicate: koekeishiya/yabai#2010 (comment)).

You are welcome to talk about your new shiny project, but you don't have to be mean about others people work, focus on your project instead.

One can understand that this performance problem is the priority for me by looking at the pinned issues.

You have received the first and the only warning about your manner of communication.

[wojciech-kulik]

Well, I'm not even going to comment on you pulling some of my random comments, because it's quite childish behavior and I'm not going to post here elaborate explaining the context.

Regarding the situation here, to give some context, I contributed to this repository, I promoted this project before anybody knew what Aerospace is, I suggested to Josean Martinez to make a YouTube video about your project, and I promoted it on reddit as well, so I believe I have some right to have an opinion on the way this project is managed, even though it's a tiny right. (btw. interesting why I don't have a contributor badge here - is it because you close pull requests instead of merging/squashing them - to avoid people being tagged as contributors?)

You, on the other hand, ignore crucial performance issues forcing people to use some 3rd party forks not even giving a clear message or plan for fixing this problem. This discussion has been already open for 3 months. To emphasize the importance of this issue, one person even donated you $500 to fix it and the only thing you did was posting a message justifying your decisions and locking the issue from further comments without any clear plan. Not a very open-source-friendly approach.

So yes, I believe I've got a right to say: "It seems like the developer is not really interested in fixing this problem", "It doesn’t seem like it’s ever going to be fixed". Sorry if it offends you, but I would feel more offended by supporting this project that much and being ignored in such a crucial bug (I'm not talking just about me). Many people stopped using this project because of this issue, even Josean Martinez who made a video about your project. So yes, it is an extremely important bug that is completely ignored by you.

One can understand that this performance problem is a priority for me by looking at the pinned issues

Pinning the issue is not a sign of something being important to you. More like avoiding more discussions in this area. This issue has been open for 3 months (and the parent issues for 1 year+) and this is your first comment here, truly important thing to you... This is my last and only message to you here.