Experimental WebGL terminal renderer

[Tyriar]

Fixes #35901
Fixes #47523
Fixes #84949

Font rendering

Notice how in the canvas line many characters get their right-side cut off (eg. ~, m, e, r):

Canvas glyphs are much more restricted because drawing them and invalidating stale glyphs is a lot more expensive. This is even worse vertically as all glyphs are clipped to remain within a row, this means some fonts don't render underscores (#35901) and some tall characters can be cut off at the top.

WebGL doesn't have this problem as every character is drawn on every frame. Currently glyphs are drawn to a canvas twice as wide as the cell with some additional vertical and horizontal padding (source) but this can be extended in the future if necessary.

Speed

Since we're leveraging the GPU at a much lower level, rendering in distributed across the GPU's many cores which results in a far lower CPU load, more responsive UI, less skipped frames and faster parsing of program output as a result. Additionally care was taken in assembling the buffers that get uploaded to the GPU to make them very fast, it's pretty much all just dealing with array buffers and numbers.

Compare the following timelines which were recorded when running ls-lR . in the vscode dir (WSL, GTX 760).

Canvas:

WebGL:

The actual rendering takes significantly less time in WebGL, notice also that the 3.96ms frame is the frame that took the longest to render in the whole timeline, the average is < 1ms.

I did some in depth measurements some time ago so they'll be a little stale, but the order of the improvement should remain around the same:

Benchmark	Canvas/dynamic (avg ms/frame)	WebGL (avg ms/frame)	Change (c/w-1)
Macbook 87x26	4.80	0.69	596% faster
Macbook 300x80	15.28	3.69	314% faster
Windows 87x26	7.31	0.73	901% faster
Windows 300x80	19.34	2.06	839% faster
Macbook 87x26 CJK	14.63	5.93	147% faster
Macbook 87x26 Emoji	27.47	19.28	42% faster

Latency between keystrokes and the graphical response should also be improved, I have not measured this though.

Improved glyph caching

Glyph caching had many drawbacks in the canvas renderer, basically only the standard 16 colors and common ascii. Anything beyond that including any combination of unicode, emoji, 256 color, true color cannot be cached and need to be drawn directly to the canvas every time. This is a particularly big deal for supporting other languages and programs that choose to draw a background color.

Canvas texture atlas after running 256 color test:

WebGL texture atlas after running 256 color test:

Additionally you will notice that there is a strategy for compressing glyphs with the WebGL renderer instead of all glyphs being the static size of a cell, allowing many more glyphs to be packed into the texture before needing to start evicting them.

Selection drawn underneath text

It's bugged me for some time that both the DOM and canvas renderers draw the selection color on top of the foreground text (#47523), there are a few reasons for this but basically performance is why they don't support this. The WebGL renderer works, additionally we can do cool things like allow a theme to set a static selection foreground color for contrast or even better support a minimum contrast level like iTerm2 (xtermjs/xterm.js#1514) which is a great accessibility feature.

Canvas (notice the blue is different in the selection):

WebGL:

WebGL has some smarts to blend the selection color into the background color if it's transparent so that existing themes won't break, now that opaque selections work. This also allows using much stronger colors if desirable since you no longer need to worry about high alpha values lowering the contrast of text:

Reliability

This can only really be verified after being out in the wild for a while but I expect WebGL to be a much better from a reliability standpoint. The canvas renderer suffers from many different issues such as several Electron/Chromium regressions in drawImage, certain configurations slowing to a crawl when rendering the text.

Less reliance on trusting the browser to do the right thing between VS Code and the GPU should improve reliability, especially since we essentially never want the browser to drop down to CPU-based canvas rendering if GPU acceleration is disabled as the DOM renderer is a far better option than that. My hope is that if webgl2 is said to be supported, it should just work and otherwise hard fail (throw) which would allow us to fallback to the DOM renderer.

Lazy loading/code splitting

One of the biggest blockers that I forced upon myself while working on this was that I didn't want the WebGL renderer to be included in the core of xterm.js, instead we worked on a new addon system that became stable in v4 and the WebGL renderer got moved into an addon. What this means is that if you do not set the renderer type to webgl then the code does not get loaded.

Early testing

Both Hyper and Terminus have been including the WebGL renderer in their products for some time. Hyper has used it as the default renderer since v3 was released back in May, Terminus has it as opt-in until transparency and ligatures work (which vscode doesn't support).

Known issues

https://github.com/xtermjs/xterm.js/issues?q=is%3Aissue+is%3Aopen+label%3Aarea%2Faddon%2Fwebgl

[Remzi1993]

@Tyriar Thank you very much for fixing this. Do you know when this gets released?

[Tyriar]

@Remzi1993 you can try it out in tomorrow's insiders by setting terminal.integrated.rendererType to experimentalWebgl

[Fmstrat]

@Tyriar What about timeframe for full release? This has been plaguing us Ubuntu users for a while now ;)

[Tyriar]

@Fmstrat depends how many problems there are with it when it goes out, I know of one major problem that I hope to fix before this release is out #85048. I'd expect it to get more and more polished until we're ready to call it stable and remove the "experimental" part (and then look at removing the canvas renderer all together). In the meantime you can just have that setting if it's working for you, I highly doubt it will get removed once it hits a stable release.

[warpdesign]

I downloaded the latest 1.41 insiders on Windows, set terminal.integrated.rendererType to experimentalWebgl but it doesn't seem to be enabled:

I tried re-starting vscode but it didn't help.

[Tyriar]

@warpdesign what's $0? There are multiple canvases that are used still, the layout should look like this where the highlighted one is the one with the webgl context:

[warpdesign]

@Tyriar This canvas doesn't have the webgl context:

[Tyriar]

@warpdesign it's webgl2