term frequency and multi highlighting

[leeoniya]

this might be more in the realm of a document search, but could be interesting to explore.

in info() or search() use a sticky regex to walk each matched haystack item with an altenation group, track frequency and have that somehow feed into sort(). also compute all highlight ranges.

also can establish some convention for haystack item parts, like heading|summary|content|footer, and ranges can be relative to partition points.

not sure how far this is worth taking. scope creep is a thing to keep in check.

[leeoniya]

matchAll could be useful for multiple out-of-order stuff, instead of sticky regex

[afonso-tsx]

is it already possible to have multi match highlighting using this internal interOr flag? @leeoniya

I tried to use it and resulted in a crash with use of .toLowerCase on undefined

[afonso-tsx]

@leeoniya Any update on this? What work is left to be done in order to have multi match highlighting?

[leeoniya]

the internals rely on 1:1 mapping between needle terms and matched parts in the haystack items, and this is unlikely to change.

probably the best i can do is to expose the internal u.prepQuery() function and leave it up to the user to use interOR flag and run the resulting regexp via .matchAll to find all terms as they appear in any order in the located matches. this could potentially find quite unexpected portions of the matches that dont correspond to the actual ranking logic.

in terms of integrating with uFuzzy.highlight(), it should be sufficient to replace the info.ranges array with the results found using matchAll and use that highlight util function normally.

https://github.com/leeoniya/uFuzzy/blob/9486cf58cd937bc15082de217febfbf793bd589a/src/uFuzzy.js#L216C21-L216C21

[punkpeye]

so how do I highlight multiple matches within a string today?

uf.search just gives one range match

[punkpeye]

It is very confusing b/c the demo shows multiple highlights, but it seems like it is not possible to do with the current library?

[leeoniya]

It is very confusing b/c the demo shows multiple highlights

this is the first time someone has found the highlighting behavior confusing.

if your needle is "my car", it will match and highlight "my fast car".

or are you refering to something different?

[punkpeye]

@leeoniya Just spent a solid hour on this until I realized that highlighting does not work as I would expect.

Haystack:

foo bar foo

Needle:

foo

I was expecting both foo instances to be highlighted.

[leeoniya]

yes, that's what this issue is about. but there are no demos that show this behavior because it is impossible for such demos to exist :)

sorry that you spent an hour before finding this.

[punkpeye]

It's alright. I just misunderstood the context of the issue and started with a bad test case.

[punkpeye]

Even though it took a hecka of a time to figure out how to make this work (due to this minor hiccup and also figuring out how to search collections with multiple fields), the search results are a night and day difference compared to fuse.js.