feat: use markdown-it-async, remove synckit

[antfu]

Description

Changed markdown-it to markdown-it-async, to eliminate the sync limitation. This makes Shiki able to load languages on-demand without synckit, which would block the thread. In the long run, make markdown-it work with async would also unlocks more potential features.

This slightly changes https://vitepress.dev/reference/default-theme-search#custom-content-renderer, which may require migration on user land. Considering it might only affect a tiny fiction of users for this advanced feature, I am not sure if VitePress considers this a breaking change. If we want to warn users for migrate, we can also enable this https://github.com/antfu/markdown-it-async#opt-in-warning

The downside of this approach is that it will breaks the Twoslash integration again - which wasn't the expected usage anyway - I have updated Shiki's docs to ask users to explicit provide the language names: https://shiki.style/packages/vitepress#setup. We should either ask users to add those languages, or always initiate them by default (which would sacrifices a bit of performance). In the meantime, I am also working on Shiki's async transformers support, which might make Twoslash integration more clean and efficient.

Linked Issues

Additional Context

---

Tip

The author of this PR can publish a preview release by commenting /publish below.

[userquin]

When building the client types, it is resolving markdown-it to cjs and it doesn't export the types (only default), check the d.ts and the d.mts files.

You can add tsconfig: 'tsconfig.json' to the client types dts plugin:

    dts({
      respectExternal: true,
      tsconfig: 'tsconfig.json',
    }),

With this change the build is fine, but we get 3 warnings:

/cc @brc-dd maybe you can comment here ;)

[userquin]

The CI is fine, check the build on Windows, the warnings are there but on Linux no.

NOTE: removing repectExternal from client types dts plugin the warnings are gone (Windows)

[brc-dd]

except for that twoslash thing, LGTM

let's check on iconify website once too. they have many mdit plugins

I'm curious though, as for most users there won't be much perf gain unless they are using too many languages. On vitepress docs: 👇

# before

pnpm docs:build:only   21.77s  user 4.45s system 118% cpu 22.102 total
max memory:                1463952 KB

# after

pnpm docs:build:only   22.08s  user 4.70s system 118% cpu 22.603 total
max memory:                1334800 KB

haven't checked the perf difference on dev though 👀

[brc-dd]

/publish

[pkg-pr-new]

npm i https://pkg.pr.new/vitepress@4507

commit: 0a40326

[antfu]

Yeah, I don't expect this to have a huge performance gain either, as the previous synckit workaround is already doing it on-demand (even tho I'd consider that approach being a bit hacky). I see it's more valuable to have markdown-it being able to be async, as it opens a lot more possibility. For example, in the future we might be able to off load the entire Shiki process to a worker for non-blocking and potentially parallelization. Shiki focused a lot on sync operations that hugely because integrations like markdown-it requires sync highlighting, and I am thinking to explore more potentials if we no logner need to be limited by that.