Giters
sail-sg
/
zero-bubble-pipeline-parallelism
Zero Bubble Pipeline Parallelism
Geek Repo:
Geek Repo
Github PK Tool:
Github PK Tool
Stargazers:
197
Watchers:
5
Issues:
12
Forks:
8
sail-sg/zero-bubble-pipeline-parallelism Issues
[QUESTION] The timing for B and W appears to be incorrect
Updated
2 days ago
Comments count
2
interleaved 1F1B seems to work better
Updated
2 days ago
Comments count
1
More general ZBV scheduling
Updated
9 days ago
Comments count
1
[QUESTION] How to avoid synchronization when using sharding
Updated
25 days ago
Comments count
3
[QUESTION] May I ask what tool was used to plot Figure 6 in paper.How can I profile bubble time in pipeline parallelism?
Updated
a month ago
Comments count
1
[QUESTION] Whether to split bw when send_backward_recv_forward is not enabled
Updated
a month ago
Comments count
4
Support sequence parallel on main branch
Updated
2 months ago
Comments count
1
Create a miniversion containing only ZB-H1 and essential changes so other megatron forks can easily integrate
Updated
3 months ago
Comments count
5
[QUESTION] Post-validation Optimizer
Closed
3 months ago
Comments count
1
Survey on implementing simple version in other codebases besides megatron-lm
Updated
5 months ago
Per stage memory control
Updated
5 months ago
Comments count
1
[BUG] Doesn't work on older pytorch versions, e.g. 1.14
Updated
6 months ago