A library to help users choose appropriate summarization tools based on their specific tasks or needs. Includes models, evaluation metrics, and datasets.
conda create -n st python=3.7
conda activate st
pip install -r requirements.txt
Imports model, initializes default model, and summarizes sample documents.
import model as st_model
model = st_model.summarizer()
documents = [
""" PG&E stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions.
The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were scheduled to be affected
by the shutoffs which were expected to last through at least midday tomorrow."""
]
model.summarize(documents)
# ["California's largest electricity provider has turned off power to hundreds of thousands of customers."]
Also, please run demo.ipynb
demo Jupyter notebook for more examples. To start demo Jupyter notebook on localhost:
jupyter notebook demo.ipynb
Import and initialization:
import model as st_model
default_model = std_model.summarizer()
bart_model = std_model.bart_model.BartModel()
pegasus_model = std_model.pegasus_model.PegasusModel()
lexrank_model = std_model.lexrank_model.LexRankModel()
textrank_model = st_model.textrank_model.TextRankModel()
All models can be initialized with the following optional options:
def __init__(self,
trained_domain: str=None,
max_input_length: int=None,
max_output_length: int=None,
):
All models implement the following methods:
def summarize(self,
corpus: Union[List[str], List[List[str]]],
queries: List[str]=None) -> List[str]:
def show_capability(cls) -> None:
def generate_basic_description(cls) -> str:
Import and initialization:
import eval as st_eval
bert_eval = st_eval.bertscore()
bleu_eval = st_eval.bleu_eval()
rouge_eval = st_eval.rouge()
rougewe_eval = st_eval.rougewe()
All evaluation metrics can be initialized with the following optional arguments:
def __init__(self, metric_name):
All evaluation metric objects implement the following methods:
def evaluate(self, model, data):
def get_dict(self, keys):
Import and initialization:
import dataset.stdatasets as st_data
This repository is built by the LILY Lab at Yale University, led by Prof. Dragomir Radev. The main contributors are Ansong Ni, Zhangir Azerbayev, Troy Feng, Murori Mutuma and Yusen Zhang (Penn State). For comments and question, please open an issue.