facebookresearch / higher

higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.

facebookresearch/higher Issues

Support of AdamW
Updated a year ago
Does higher work with hugging face Adafactor?
Updated a year ago2
Does higher work with huggingface (hugging face, HF) models? e.g. ViT?
Updated a year ago2
metaclass issue with fastai
Updated a year ago4
CUDA out of memory
Updated a year ago
once_differentiable
Updated a year ago1
More complete example please?
Closed 3 years ago13
In DifferentiableAdam, sqrt() is non-differentiable at zero
Updated 2 years ago6
grad clip correctness
Updated 2 years ago
Using higher for hyperparameter optimization
Updated 2 years ago1
Non scalar loss
Updated 2 years ago1
Why DifferentiableOptimizer detaches parameters when track_higher_grads = False?
Updated 2 years ago7
when do we divide by met_batch_size?
Updated 2 years ago1
Potential bug with first order MAML using only higher, setting track_higher_grads = False leads to .grad field to not be populated and be None, is that a bug?
Updated 2 years ago4
Intialize Differentiable Optimizer with non-leaf tensort
Updated 2 years ago1
higher for dpt architectures?
Updated 2 years ago2
How to Update/Optimize subset of params
Updated 2 years ago2
When will higher allow the use of DDP (distributed data parallel)?
Updated 2 years ago4
Would making the gradient "data" by detaching them implement first order MAML?
Updated 2 years ago
Mixed precision training
Updated 2 years ago
higher for Reinforcement Learning?
Closed 3 years ago2
second-order deravative
Updated 3 years ago
Is there data leakage in the maml-omniglot example?
Updated 3 years ago6
Setting up a templates got gitissues that invite users to post to pytorch forum
Updated 3 years ago
How does one return an adapted model without using the context manager?
Updated 3 years ago8
variables work outside of context manager scope
Updated 3 years ago2
How to evaluate model without gpu memory issues?
Updated 3 years ago1
Memory not freed when moving out of scope?
Updated 3 years ago1
maml omniglot - finetuning / test
Updated 3 years ago5
What does the warning flag in testing for maml mean? what does it mean to "fine tune for testing"?
Updated 3 years ago
Does higher work with DDP only test time evaluation (no training just testing)?
Updated 3 years ago
Is higher compatibale with distributed data parallel DDP ?
Closed 3 years ago8
Is the higher library compatible with pytorch's distributed RPC?
Updated 3 years ago6
implementing MAML with MiniImageNet
Updated 3 years ago
Tracking higher-order grads when arbitrarily combining submodules of functional module
Updated 3 years ago
The fmodel can't transport the running_mean and running_var of BN layer to the original model
Updated 3 years ago
torch.optim.AdamW not in the list of supported optimizers
Updated 3 years ago2
Copy diffopt state to original optimizer
Updated 3 years ago2
dependency installation issues with requirements.txt
Updated 3 years ago1
How to double check that 2nd order grads are being used
Updated 3 years ago1
How to use multiple optimizer in the inner loop?
Updated 3 years ago1
How does one execute an individual higher nn patched module?
Updated 3 years ago9
Discrepancy between code & manual calculation of parameter gradient w.r.t older version of parameter
Closed 4 years ago2
is the accumulation of gradient done right, where do we divide the accumulator by number of tasks?
Closed 4 years ago4
AttributeError: 'NoneType' object has no attribute '_parameters'
Closed 4 years ago2
MAML++ implementation?
Closed 4 years ago3
DifferentiableOptimizer not setting self.param_groups to be the same as reference optimizer
Updated 4 years ago3
How to train a model inside an innerloop context without higher order gradients?
Closed 4 years ago4
Documentation for the MonkeyPatched module class
Updated 4 years ago
Understanding the higher and non-higher code snippets
Updated 4 years ago