InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
Home Page:https://internevo.readthedocs.io/zh-cn/latest/?badge=latest
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
Cerberous opened this issue 6 months ago · comments
现在Internevo代码中的tflops直接按照公式计算，但是当使用tp或者pp的时候模型被切开了，导致tflops不准确
官方镜像代码
No response
@li126com 帮忙看看
我们这里计算的tflops指的是整体的而非 per GPU，所以不需要考虑tp pp这些，类似于megtron中在算total_flops时也不需要考虑
