bigscience-workshop / bigscience

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

bigscience-workshop/bigscience Issues

Why is deepspeed enabled in the Bloom training script?
Updated 7 months ago
Zero_Stage=1 results in higher TFLOPS?
Updated a year ago1
why 384（12*2*16） will be the first time all pipeline stages be filled
Closed a year ago
Where can I download the training script for bloom-7b1?
Updated a year ago
How to get train-splits.txt and valid-splits.txt before training tr11-176B-ml
Updated a year ago1
Files for bias evaluation
Updated a year ago
where can we get a bloomz-7b1 finetuned checkpoint
Updated a year ago
eval opt-175B
Updated 2 years ago1
Running Bloom
Updated 2 years ago5
Requirements to perform inference over the BigScience Bloom model
Updated 2 years ago
mC4 sampling & pre-processing
Updated 2 years ago1
What is the number of epochs of the final training?
Updated 2 years ago
About training data for 1B3 models
Closed 2 years ago
Sharing the 1.3B-Pile@300B model
Updated 2 years ago
Wrong tokenizer path in big model config
Closed 2 years ago1
Is the 13B - unmodified Megatron gpt2 - baseline available? ( tr1-13B-base)
Updated 2 years ago1
can you share the slurm.conf you are using?
Updated 2 years ago3
make a back up for final training data
Closed 2 years ago1
Fill in request for the second half of compute
Closed 3 years ago