12-NLP Tools.md

• Natural Language Processing Best Practices & Examples ⭐⭐⭐⭐⭐

  https://github.com/microsoft/nlp

Datasets

• Alphabetical list of free/public domain datasets with text data for use in Natural Language Processing (NLP) ⭐⭐⭐⭐⭐

  https://github.com/niderhoff/nlp-datasets

• Collections of Chinese NLP corpus

  https://github.com/OYE93/Chinese-NLP-Corpus

• hanzi_char_featurizer

  汉字字符特征提取器 (featurizer)，提取汉字的特征（发音特征、字形特征）用做深度学习的特征

  https://github.com/howl-anderson/hanzi_char_featurizer

Tools

• https://spacy.io/universe all related NLP projects !!! ⭐⭐⭐⭐⭐

  https://irl.spacy.io/ - conference spacy related !

• prodigy

  https://prodi.gy/

  Annotation Flowchart: Named Entity Recognition

  https://prodi.gy/docs/workflow-text-classification

• doccano

  Open source text annotation tool for machine learning practitioner. https://doccano.herokuapp.com

  https://github.com/chakki-works/doccano

• AlpacaTag

  AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging http://inklab.usc.edu/AlpacaTag/

  https://github.com/INK-USC/AlpacaTag

• chazutsu

  The tool to make NLP datasets ready to use https://medium.com/chakki/how-to-load…

  https://github.com/chakki-works/chazutsu

• scattertext

  Beautiful visualizations of how language differs among document types.

  https://github.com/JasonKessler/scattertext

• textpipe

  Textpipe: clean and extract metadata from text

  https://github.com/textpipe/textpipe

• pattern

  Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization. http://www.clips.ua.ac.be/pages/pattern

  https://github.com/clips/pattern

• TextBlob

  Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more. https://textblob.readthedocs.io/

  https://github.com/sloria/textblob

• PyNLPl

  PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotatation).

  https://github.com/proycon/pynlpl

• NLTK

  NLTK -- the Natural Language Toolkit -- is a suite of open source Python modules, data sets, and tutorials supporting research and development in Natural Language Processing.

  For documentation, please visit nltk.org.

  https://github.com/nltk/nltk

• spaCy

  Library Architecture - https://spacy.io/api (pipeline)

  💫 Industrial-strength Natural Language Processing (NLP) with Python and Cython https://spacy.io

  https://github.com/explosion/spaCy

  https://github.com/explosion/thinc 🔮 spaCy's Machine Learning library for NLP in Python

  https://github.com/chartbeat-labs/textacy NLP, before and after spaCy https://chartbeat-labs.github.io/text…

• polyglot

  Multilingual text (NLP) processing toolkit http://polyglot-nlp.com

  https://github.com/aboSamoor/polyglot

• This is a list of NLP tools for various purposes.

  http://www.phontron.com/nlptools.php

Architect

• nlp-architect

  A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks http://nlp_architect.nervanasys.com/

  https://github.com/NervanaSystems/nlp-architect

Application

NLU (Natural Language Understanding)

• Rasa

  💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants https://rasa.com/docs/

  https://github.com/RasaHQ/rasa

• Rasa_NLU_Chi

  Turn Chinese natural language into structured data 中文自然语言理解

  https://github.com/crownpku/Rasa_NLU_Chi

• rasa_nlu_gq

  turn natural language into structured data(支持中文，自定义了N种模型，支持不同的场景和任务)

  https://github.com/GaoQ1/rasa_nlu_gq

• Clause

  🏇 Chatopera语义理解服务 https://bot.chatopera.com

  https://github.com/chatopera/clause

• Snips

  Snips Python library to extract meaning from text https://snips-nlu.readthedocs.io

  https://github.com/snipsco/snips-nlu

Tensorflow

• Kashgari

  Kashgari is a Production-ready NLP Transfer learning framework for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.

  https://github.com/BrikerMan/Kashgari

• deepnlp

  Deep Learning NLP Pipeline implemented on Tensorflow. Following the 'simplicity' rule, this project aims to use the deep learning library of Tensorflow to implement new NLP pipeline. You can extend the project to train models with your own corpus/languages. Pretrained models of Chinese corpus are distributed. Free RESTful NLP API are also provided. Visit http://www.deepnlp.org/api/v1.0/pipeline for details.

  https://github.com/rockingdingo/deepnlp

PyTorch

• fastNLP

  fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.

  已复现的论文:

  Star-Transformer Biaffine CNNText ...

  https://github.com/fastnlp/fastNLP

• AllenNLP

  An open-source NLP research library, built on PyTorch. http://www.allennlp.org/ AllenNLP: A Deep Semantic Natural Language Processing Platform

  https://github.com/allenai/allennlp

• pytext

  A natural language modeling framework based on PyTorch https://fb.me/pytextdocs

  https://github.com/facebookresearch/pytext

• flair

  A very simple framework for state-of-the-art Natural Language Processing (NLP)

  https://github.com/zalandoresearch/flair

• pytorch-transformers

  👾 A library of state-of-the-art pretrained models for Natural Language Processing (NLP) https://huggingface.co/pytorch-transf…

  https://github.com/huggingface/pytorch-transformers

• FARM

  Fast & easy NLP transfer learning for the industry. Harvesting models for practical use cases. 🏡 https://farm.deepset.ai

  https://github.com/deepset-ai/FARM

• decaNLP

  The Natural Language Decathlon: A Multitask Challenge for NLP The Natural Language Decathlon: Multitask Learning as Question Answering

  https://github.com/salesforce/decaNLP

School

• https://nlp.stanford.edu/software/

• http://nlp.cs.berkeley.edu/software.shtml

Java

• opennlp

  https://github.com/apache/opennlp