Giters

tfjgeorge / nngeometry

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

Home Page:https://nngeometry.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support of ParameterList

arthurdouillard opened this issue · comments

commented

Hello,

I'm trying to support FIM with PMatDiag for my model that has some nn.ParameterList but:

Traceback (most recent call last):
  File "run.py", line 381, in <module>
    main(train_scenario, test_scenario, args)
  File "run.py", line 191, in main
    device='cuda'
  File "/local/douillard/conda_env_continualexp/lib/python3.7/site-packages/nngeometry/metrics.py", line 131, in FIM
    layer_collection = LayerCollection.from_model(model)
  File "/local/douillard/conda_env_continualexp/lib/python3.7/site-packages/nngeometry/layercollection.py", line 46, in from_model
    raise Exception('I do not know what to do with layer ' + str(mod))
Exception: I do not know what to do with layer ParameterList(
    (0): Parameter containing: [torch.FloatTensor of size 50x5]
    (1): Parameter containing: [torch.FloatTensor of size 50x5]
)

Could you support it please?

Thank you already for your work and codebase! ❤️

cc @TLESORT

commented

Hi, thanks for using NNGeometry and reporting a feature request!

I am not so familiar with ParameterList objects, but I am guessing that they are used to define custom layers ? Unfortunately we do not support custom layers, since we have to explicitely reimplement the backward computation of individual gradients for each layers, and we cannot guess how to do this for custom layers.