google-research / bigbird

Transformers for Longer Sequences

https://arxiv.org/abs/2007.14062

google-research/bigbird Issues

Model for genomic sequences
Updated 6 months ago
Variable error with the full_bigbird_mask method in the multi head attention class
Updated 6 months ago
Pre-trained model for genomic sequences
Updated a year ago9
the versions of all libraries in the deployment environment?
Updated a year ago
reproduce arxiv classification task
Updated a year ago1
Why is BigBird Pegasus/Pegasus Repeating the Same Sentence for Summarization?
Updated 2 years ago1
I've added bigbird's attention to my model, but not seeing a decrease in memory
Updated 2 years ago5
Any plan to provide chinese pretrain model ?
Updated 2 years ago
Export predictions for each example
Updated 2 years ago3
Are encoder and decoder both implemented with sparse attention? How long is the verified output length for the decoder?
Updated 2 years ago
TFDS Custom Dataset Issue - normalizer.cc(51) LOG(INFO) precompiled_charsmap is empty. use identity normalization.
Closed 2 years ago1
Differences between ETC and BigBird-ETC version
Updated 2 years ago
How is Prior Arts, which can only accept short text input, evaluated on long text datasets.
Updated 2 years ago
code error in version of tensorflow?
Updated 2 years ago
Learning rate mentioned in paper vs run_summarization.py
Updated 3 years ago
What's the difference of bigbr_base and bigbr_base_tf2 at the gs://bigbird-transformer/pretrain ?
Updated 3 years ago
Error in PubMed evaluation using run_summarization.py
Updated 3 years ago3
How can we finetune the pretrained model using tfrecord files?
Closed 3 years ago1
Precision equals Recall in run_classifier.py script run.
Updated 3 years ago1
Why ``last_idx`` set to 1024 even when sequence length goes upto 4096?
Updated 3 years ago
detail about warm start from RoBERTa’s checkpoint.
Updated 3 years ago
Error in run_classifier.py for attention_type=simulated_sparse
Updated 3 years ago
Preprocessing code for the arxiv classification dataset.
Updated 3 years ago1
Couldn't able to save and load the model after finetuning
Updated 3 years ago1
bug in line-494 of script- run_pretraining.py
Updated 3 years ago
Unconditional assert False in bigbird/core/utils.py
Updated 3 years ago
Pegasus variables mapping
Updated 3 years ago1
Is it valid to train on GRCh38.p13 human reference instead of GRCh37 ?
Updated 3 years ago
Would you like to release the code about how to train a bigbird with other language
Closed 3 years ago1
I want to know d.map("preprocess function",... ) processing
Updated 3 years ago
Preprocessing code for TriviaQA dataset
Closed 3 years ago1
Roberta Training
Updated 3 years ago4
Question about pre-trained weights
Closed 3 years ago3