This repository has been archived by the owner on Oct 10, 2024. It is now read-only.

Implement Word Dictonary / BagOfWords #19

Open

aron-bordin opened this issue Oct 7, 2017 · 0 comments

Labels

enhancement help wanted medium

Member

aron-bordin commented Oct 7, 2017 •

edited

Loading

Receives text (or a list of texts) as input, and a tokenizer, and generates a word dictionary.

implement with dask for multiprocessing/threading speedup, or implement with C (discuss what is the best option).
limit the number of elements in the dict
set the minimum occurrence
can be used as a bag of words, keeps the freq of each term in the corpus.

aron-bordin added the enhancement label

aron-bordin added the medium label

aron-bordin added the help wanted label

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.