THUDM / SwissArmyTransformer

SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.

https://THUDM.github.io/SwissArmyTransformer

THUDM/SwissArmyTransformer Issues

请问断点续训应该如何设置
Updated a month ago6
transfer_param.py 转换vincuna hf模型成sat模型报错
Updated a month ago12
Fail to load random states from checkpoints saved
Updated a month ago2
How to install a model to the right path?
Closed 2 months ago1
NO cogagent？
Updated 2 months ago2
ModuleNotFoundError: No module named 'localAttention'
Updated 2 months ago
SwissArmyTransformer可以读bin权重文件吗？visualglm-6b项目里就没见pt文件，只有bin。难以微调
Closed 3 months ago5
“No backend type associated with device type cpu” when run cli_demo_sat.py
Updated 3 months ago5
如果想绕过deepspeed做finetune，可以在train的时候直接model.step()来实现吗？
Updated 3 months ago1
Using CogVLM - KeyError (MODEL_URLS) - Google Colab
Closed 3 months ago1
MixtralMlpMixin()这个函数里面moe只是计算专家的logits但是没看到分发逻辑
Updated 3 months ago1
AutoModel.from_pretrained()里面不能加载hf版本的权重吗
Updated 4 months ago1
AutoModel.from_pretrained()里面不能加载hf的权重吗
Closed 4 months ago
怎么从断点恢复微调训练
Updated 4 months ago1
请问针对样本数量不均衡的数据集怎么做样本均衡呢
Updated 4 months ago1
Questions about your LoRA codes
Closed 4 months ago7
Is sat suuport saving checkpoint by using fp16 or bf16?
Updated 4 months ago5
deepspeed 分布式训练 loss nan or inf
Updated 4 months ago1
不支持流式dataset
Closed 4 months ago2
How to load and initialize llama2 models downloaded from Huggingface
Closed 4 months ago2
单机多卡训练时内存占用过高
Closed 4 months ago2
deepspeed分布式训练出现sat ValueError inconsistent
Updated 5 months ago1
How to embed video encoder module from pytorch？
Updated 6 months ago3
Can you help to confirm if chatglm3 model is same as GPT or it's original from GLM architecture?
Closed 6 months ago3
请问如何使用hf加载icetk_glm_130B的tokenizer和GLM130B的模型？
Closed 6 months ago6
FileLock - out of date?
Closed 6 months ago1
ore.exceptions.ResponseStreamingError
Updated 7 months ago1
ModuleNotFoundError: No module named 'SwissArmyTransformer'
Updated 7 months ago6
Cannot use torch.compile with SAT
Updated 7 months ago
Window 安装错误
Updated 8 months ago2
使用微调时，由于数据集过大，内存不够如何处理？
Closed 9 months ago1
请教一个问题，使用mp_size=2时的loss应该怎么写
Updated 9 months ago1
模型并行的方式进行lora方式的finetuning要怎么设置呢
Updated 9 months ago5
huggingface版本的visualglm在前向传播报错，Exception: cuda rng state model-parallel-rng is not added
Updated 9 months ago1
During BERT decoding, past_key_values is used to accelerate calculation. Do we have a similar implementation?
Updated 9 months ago1
使用0.2.x版本时报错
Updated 9 months ago1
测试源码中给的qlora.py报错
Updated 10 months ago7
sat.arguments.get_args failed to handle the "-h" option
Updated 10 months ago1
Request for documentation: the relationship between "Swing Army Transformer" and Nvidia's "FasterTransformer"
Closed 10 months ago1
请问dataloader能shuffle吗？
Closed 10 months ago2
0.2.12 release版本的源码在哪个分支？
Closed 10 months ago1
SAT Tokenizer 地址挂了
Updated a year ago1
在镜像中安装报错
Updated a year ago2
sat 和 transformers & huggingface hub 可以无缝集成吗？
Updated a year ago1
怎样使用DeepSpeed的offload功能降低显存占用？
Updated a year ago
安装时报错
Updated a year ago1
AutoModel加载模型时报错
Updated a year ago1
可以在win或mac本地进行吗？
Updated a year ago1
pypl 清华源，没有swissarmytransformer
Closed a year ago2
AssertionError: data parallel group is not initialized
Closed a year ago3