zohebk / text_summarization

various ways to summarize text and for multiple languages using Python Library. Created on 13 Mar 2018

Created on : 13 Mar 2018

NOTE: pytldr and pyteaser doesn't work for python>2.7.
Summarize any language text

run MultiLanguageSummarizer.py

Extractive text summarization

various ways to summarise text using the libraries available for Python

1. pyteaser
2. sumy
3. gensim
4. pytldr
5. XLNET
6. BERT
7. GPT2

INSTALL

pip install sumy
pip install gensim
pip install pyteaser
pip install pytldr
pip install bert-extractive-summarizer
pip install spacy==2.0.12
pip install transformers==2.2.0

Pyteaser

Pyteaser has two function:
Summarize: that takes title and text and summarizes them
SummarizeURL: that takes the url and summarizes the content of the url

Sumy

Summy has various preprocessing libraries and summarizer libraries
sumytoken: for tokenizing the text
get_stop_words: to remove the stop words from the text
stemmer: to stemp the words
LexRankSummarizer: summarizes based on lexical ranking
LsaSummarizer: summarizes based on semantic
LuhnSummarizer: summarizes based on Luhn's algorithm

Gensim

gensim has a summarize library which can be imported and used directly.

pytldr

pytldr is also like sumy where they have various nlp libraries like tokenizer.
Here we have used TextRankSummarizer, RelevanceSummarzer, LsaSummarizer from pytldr

PS:

If you are using python2, please run main.py from "for_python2" folder
elif you are using python3, please run main.py from "for_python3" folder
else you can test by running "various_ways_to_summarize.py" or the notebook named as "Text Summarizer Notebook.ipynb"