InternLM / InternEvo

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

https://internevo.readthedocs.io/zh-cn/latest/?badge=latest

InternLM/InternEvo Issues

[QA] loong train 支持packed_sample_into_one=false吗
Updated 2 months ago1
[QA] 如何进行单卡微调的，需要调整那些设置
Updated 2 months ago
[Bug] There will be timeout in some cases.
Closed 3 months ago1
[Feature] MoE模型里稠密层和专家层zero和并行的解耦
Updated 4 months ago
[Feature] 不使用memory pool
Updated 4 months ago1
[Bug] There is an error in training : built-in model should inherited from BaseModel
Closed 4 months ago1
[Feature] 优化ce_loss计算
Closed 4 months ago
[QA] 代码中涉及到的字符串比较，整改为枚举类型比较
Closed 4 months ago
[QA] 梳理load_hf_llama_pretrained_weights相关代码逻辑，清理无用代码
Closed 4 months ago
[Bug] 昇腾910安装internLM环境时报错需要nvcc
Closed 4 months ago2
[Bug] 910B已知BUG和解决情况
Closed 4 months ago
[Bug] Import Error: Import "deeplink_ext.internlm_ops" could not be resolved
Closed 4 months ago1
[QA] check import system var at the start of training
Updated 4 months ago
[Feature] how to finetuning lora
Updated 5 months ago1
[Bug] RuntimeError: [3] is setting up NCCL communicator and retrieving ncclUniqueId from [0] via c10d key-value store by key '0', but store->get('0') got error: Socket Timeout
Updated 5 months ago
[QA] Internevo是否支持tied_embedding?
Updated 5 months ago3
[Bug] 仅支持了GShard模式的MoE模型转huggingface
Updated 5 months ago
[Bug] 训练bf16 infer fp16出现NaN
Updated 5 months ago
[QA] 使用InternLM pretrain llama2-7b之后，怎么用训练之后的模型做推理？
Closed 5 months ago1
[QA] InternEvo能否load预训练llama2的参数
Updated 5 months ago4
[Bug] 使用internevo训练，转换成hf模型用opencompass测试时候有一定概率会nan
Updated 6 months ago2
[Bug] 好像没有把internevo的MoE权重转换成huggingface版本的脚本？
Updated 6 months ago8
[Bug] TFLOPS计算不准
Updated 6 months ago2
[Bug] 用MoE训练的时候tflop超级低
Updated 6 months ago4
[QA] 用Internevo已经训练出来了一个7B模型，如何用这个internevo权重跑MoE？
Updated 6 months ago1
[QA] Internevo这个框架里面MoE支持expert parallel嘛？
Updated 6 months ago1
[Bug] 昇腾910微调internLM报错
Updated 6 months ago3
[Bug] meta数据load进来内存过大，训练过程中内存持续增长
Closed 6 months ago7
[QA] llama2-7b模型配置和原生疑似不一致
Closed 6 months ago
[Bug] 训练报错indexSelectLargeIndex: block: [604,0,0], thread: [47,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
Closed 6 months ago1
[QA] 关于使用张量并行或流水线并行的模型切分与合并问题
Updated 6 months ago
[Bug] internlm docker image issue
Closed 6 months ago2
[Bug] AssertionError: Only flash cross entropy support parallel_output
Updated 6 months ago1
[Bug] No module named 'internlm.model.ops.fusion_ops_import_helper'
Closed 6 months ago1
[Bug] npu训练生成的模型转换为hf格式报错
Closed 6 months ago3
[Bug] TypeError: record_current_batch_training_metrics() missing 1 required positional argument: 'very_begining_time'
Closed 7 months ago1
Add tool for data cleaning
Updated 7 months ago
[Bug] IndexError: too many indices for tensor of dimension 2
Closed 7 months ago1
[Bug] The task keeps hanging in launch stage
Closed 8 months ago1
[Bug] record_current_batch_training_metrics() got an unexpected keyword argument 'update_panel'
Closed 9 months ago1
[Feature] Should we remove other dependency of flashattention?
Updated 9 months ago
[Bug] RMSNormTorch should also setattr (param, IS_REPLICA_ZERO_PARALLEL, True)
Closed 9 months ago
[Bug] 使用moe的config微调报错
Updated 9 months ago1
[Feature] Support customized model size for training
Updated 9 months ago
[Feature] use consist way for get device
Closed 9 months ago1
[Bug] support profiling on NPU
Closed 9 months ago1
[Feature] Recommend to replace internlm1.1 with internlm2 in web-demo
Updated 9 months ago
[Bug] the usage "x.is_cuda" is not recommended, since we have both GPU and NPU
Updated 9 months ago1
[Bug] do not use torch.cuda.current_device() as device, since it only retures an int
Updated 9 months ago3
[Feature] do not export the global variable internlm_accelerator
Closed 9 months ago1