Giters
OpenLMLab
/
LOMO
LOMO: LOw-Memory Optimization
Geek Repo:
Geek Repo
Github PK Tool:
Github PK Tool
Stargazers:
956
Watchers:
13
Issues:
70
Forks:
69
OpenLMLab/LOMO Issues
请问adalomo可以支持用transformer中的trainer训练么?或者未来有可能实现么?
Closed
24 days ago
Comments count
20
One confusion about the LOMO paper
Closed
3 months ago
Comments count
1
wandb permission
Closed
a year ago
Comments count
4
MOE and Custom Finetuning
Closed
4 months ago
installable package
Closed
5 months ago
Comments count
2
Serious conclusion: LOMO does not significantly reduce GPU memory usage!
Updated
6 months ago
Comments count
17
为什么LOMO并没有火起来呢?
Updated
6 months ago
Comments count
5
eval environment for opencompass
Updated
7 months ago
Comments count
4
How to load a 65B model on 24G GPU memory?
Closed
7 months ago
Comments count
37
实测LOMO++ deepspeed zero2 7b qlora llama 显存占用 感觉比正常的qlora + deepspeed zero2 显存占用大1倍
Updated
7 months ago
Comments count
1
Can you provide detailed dependency versions?
Updated
7 months ago
Evaluation of Fine-tuned model with Adalomo
Closed
8 months ago
Comments count
1
adalomo在使用chatglm2模型出现错误
Updated
8 months ago
Comments count
2
adalomo 试图在 llama 2 70b 模型训练时出现 NCCL communicator 这类超时的错误
Updated
8 months ago
Comments count
5
Reproduce the results for LOMO
Closed
9 months ago
Comments count
2
adalomo optimizer error
Closed
9 months ago
Comments count
15
Instructions for evaluation datasets
Closed
9 months ago
Comments count
3
model merge error
Closed
9 months ago
Comments count
18
Mistral Support
Closed
9 months ago
Comments count
1
type object 'torch._C._distributed_c10d.ReduceOp' has no attribute 'AVG'
Closed
9 months ago
Comments count
4
ModuleNotFoundError: No module named 'rich' after ' python -m pip install rich'
Closed
9 months ago
Comments count
1
cannot find adalomo class
Closed
9 months ago
Comments count
2
a bug found in save_model of LOMOTrainer
Updated
9 months ago
Comments count
10
Runtime error on 2nd epoch, trying lora only
Updated
9 months ago
hook函数之所以再需要额外操作一次,那里的描述我觉得有些不妥,也或者我理解有问题
Closed
10 months ago
Comments count
1
Customized loss value
Updated
a year ago
Comments count
4
LlaMA-7B + LoRA在16GB的V100上OOM
Updated
a year ago
Comments count
2
选取bloom-1b7作为model和wic作为数据集出现IndexError: tuple index out of range
Updated
a year ago
Comments count
1
请问自定义Dataset只能是classification的数据集吗
Updated
a year ago
Comments count
1
小学习率问题
Updated
a year ago
Comments count
1
Is LOMO a concurrent work of the official implementation?
Closed
a year ago
Comments count
4
LOMO是否支持bfloat16模型的训练?
Closed
a year ago
Comments count
6
CLIP梯度和梯度overflow的影响
Updated
a year ago
Comments count
2
Memory Usage continues to grow
Updated
a year ago
Comments count
5
batch size开2后一直提示gradient overflow。。
Closed
a year ago
Comments count
16
Functions to measure the memory usage
Closed
a year ago
Comments count
1
请教个问题,LLM 训练会存在 micro-batch 之间需要累积梯度的场景,这种场景也会有优化吗?
Closed
a year ago
Comments count
1
公式4疑问
Closed
a year ago
Comments count
1
关于代码理解和显存占用的问题
Updated
a year ago
Comments count
1
llama-33B/llama-65B均报OOM,8*V100跑不起来怎么回事呢?
Updated
a year ago
Comments count
7
关于微调llama-65b的疑问
Closed
a year ago
Comments count
2
Performance Model after Full Fine-tuning by LOMOTrainer
Updated
a year ago
Comments count
9
How to calculate the used GPU memory for each part as in the paper?
Updated
a year ago
Comments count
2
LORA+LOMO distributed learning
Closed
a year ago
Comments count
2
LOMO+QLoRA简单更改后的报错
Closed
a year ago
Comments count
7
Key Error: LOCAL_RANK
Closed
a year ago
Comments count
1
about torch.stack(self.grad_norms)
Updated
a year ago
Comments count
3
我使用了Resnet50+LOMO优化器,使用cpu去跑,系统内存相比sgd 没有任何变化,请问合理吗
Closed
a year ago
Memory consumption first grows up then falls down.
Updated
a year ago
Comments count
3
Some confusion about the method of the paper
Updated
a year ago
Comments count
3
Previous
Next