google-research / tuning_playbook

A playbook for systematically maximizing the performance of deep learning models.

Repository from Github https://github.comgoogle-research/tuning_playbook

google-research/tuning_playbook Issues

Update for Transformers/LLM/SLM
Updated a month ago
Question about optimizing away nuisance hyperparameters and the need for a secondary validation set
Updated 3 months ago
"trial" or "study"
Updated 6 months ago
Missing image "have_we_sampled_enough.png" in FAQ section
Updated 7 months ago
Request to cover "2-3 weeks upper bound with data-freeze"
Updated 8 months ago
Request to cover "Insensitivity to tuning" in the playbook
Updated 8 months ago
Clarification is needed in the chapter "How should Adam’s hyperparameters be tuned?"
Closed 8 months ago1
Tuning
Closed 9 months ago
Utilize `Hyperband`/`ASHA` scheduler? (Willing to PR)
Updated 9 months ago3
Why avoid gradient accumulation?
Updated 9 months ago5
About contributing a paper related to choosing optimizer
Closed 9 months ago3
Consider recommending scipy.stats.qmc for quasi-Monte Carlo sequence generation
Updated a year ago4
Question about training steps
Closed a year ago2
How to examining the training curves about a multi-task learning model?
Closed a year ago2
`training throughput` may not equal to `time per step`
Closed a year ago2
Typo in the order of applying Norm and function
Closed a year ago1
Question: Time Series Missing Data?
Closed a year ago2
question about the performance of tuning
Closed a year ago1
Topic about finetune checkpoint
Closed a year ago1
Gradient checkpointing
Closed a year ago1
Many picture links expired.
Closed a year ago2
[BUG] In Choosing the goal for the next round of experiments
Closed a year ago1
Evidence for Norm(x + f(x)) causing issues?
Closed a year ago3
Delphi process
Closed a year ago1
Conversion to flowchart
Closed a year ago1
Adding some caveats about hyperparameter tuning at large batch sizes
Closed a year ago3
Add Chinese version of this document
Closed a year ago7
add turkish language translation version of the book
Closed a year ago2
Publish playbook on GitHub Pages
Closed a year ago1
A small question about the Momentum equation section
Closed a year ago1
Typo: Missing parenthesis in nesterov momentum
Closed a year ago3
Provide (open source) scripts for visualizations in the playbook
Closed a year ago1
Wrong link in url under #citing
Closed 2 years ago1
吐槽
Closed 2 years ago2
あつもか
Closed 2 years ago1
Why "Normalization should be the last operation before the residual" ?
Closed 2 years ago2
All benefits of using a larger batch size assume the training throughput increases?
Closed 2 years ago
Throughput of gradient accumulation
Closed 2 years ago1
Add multi-language versions of this playbook
Closed 2 years ago1
Deep learning tuning playbook
Closed 2 years ago1
Any plan for multi-langauge translated version?
Closed 2 years ago1
Wrong link in table of contents
Closed 2 years ago1
Add related paper about choosing optimizer
Closed 2 years ago