Tagging sequence markup for entity extraction

[hoagy-davis-digges]

How can I use a tagging sequence such as IOB with snorkel features?

[jason-fries]

Hi @hoagy-davis-digges

The easiest way to implement IOB tagging is to treat every word as a candidate and then use categorical variables, similar to the tutorial at: https://github.com/HazyResearch/snorkel/blob/master/tutorials/advanced/Categorical_Classes.ipynb

Here your labeling functions would output labels in the space B, I , O, abstain. This approach doesn't model dependencies between labels (like a CRF would), but not all problems actually need this.

We're working on a formal entity tagging paper, so these types of tasks will be easier to work with in Snorkel in the near future.

[hoagy-davis-digges]

Awesome, thanks!

[ajratner]

Closing for now--will be accessible via the "Q&A" link in README--but feel free to re-open!

[jbkoh]

Hi @jason-fries
Is there any update on the entity tagging with Snorkel you mentioned above? It'd be awesome if I could try it out for my use case. I've been doing research on entity resolution in metadata of buildings. I can also contribute if there is a room for it.

Thanks for the great project!

[thak123]

Any update in this regard?

[mpgxc]

Any update in this regard?

[jason-fries]

Hi @mpgxc,

The general idea behind sequence labeling with off-the-shelf Snorkel is outlined in Issue #1254. More technical details are available in our manuscript and our demo notebook for weakly supervised NER at https://github.com/som-shahlab/trove/