You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Possible ways to take advantage of this are to provide some kind of hook for user code to provide it's own UTF8 validation, or a compile time option to specify a UTF8 validation function as a dependency.
The text was updated successfully, but these errors were encountered:
It seems to use vectorized (SIMD) instructions, i would say it goes a bit to far to have this kind of optimization. The UTF8 validation overhead is only the tiniest percentage of the whole workload. Not sure if optimization of this would give you any noticable performance gain. I wonder why boost locale does not have a validation function and select an optimized version based on CPU architecture.
Just making the project aware of this faster algorithm. https://lemire.me/blog/2020/10/20/ridiculously-fast-unicode-utf-8-validation/
Possible ways to take advantage of this are to provide some kind of hook for user code to provide it's own UTF8 validation, or a compile time option to specify a UTF8 validation function as a dependency.
The text was updated successfully, but these errors were encountered: