AttributeError: 'XLNetModel' object has no attribute 'embeddings'

[Nils Feldhus]

The following error was raised for "mnli-xlnet-lgxa":

Traceback (most recent call last):
  File "run_explainer.py", line 57, in <module>
    explainer = getattr(thermex, f'Explainer{config["explainer"]["name"]}').from_config(config=config)
  File "/home/feldhus/thermostat/src/thermostat/explainers/grad.py", line 71, in from_config
    layer=res.model.base_model.embeddings)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1094, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'XLNetModel' object has no attribute 'embeddings'

[Robert Schwarzenberg]

seems like you search for embeddings but i use transformer.word_embedding

https://github.com/rbtsbg/gxai/blob/c183d48a8c871ea38c9529a0d24017bcccb4f833/explain.py#L164

since XLNet / LIG was not part of the paper, I did not check this in depth, so please double check it

also keep in mind that i might have worked with another version and that this could be deprecated

[Nils Feldhus]

Thanks! I think inserting a case distinction for XLNet will suffice.
It's just that I think it's weird that XLNet seems to be the only model which does not have the embeddings attribute.

[Robert Schwarzenberg]

I did not find it then and word_embedding is my best guess

[Robert Schwarzenberg]

sorry closed by accident

[Robert Schwarzenberg]

so, i investigated the xlnet code. the word embeddings layer does not contain positional and segment embeddings, but it appears to be what we suspect it to be.

https://github.com/huggingface/transformers/blob/80d712fac6ccae308a2f408ebbc0c4d8c482d509/src/transformers/models/xlnet/modeling_xlnet.py#L1189it

returns the word embeddings

later, the word embeddings are forwarded with the other embeddings, i.e. positional, segment ...

https://github.com/huggingface/transformers/blob/80d712fac6ccae308a2f408ebbc0c4d8c482d509/src/transformers/models/xlnet/modeling_xlnet.py#L1249

[Nils Feldhus]

Wow, that automated message above this is really neat!
I don't really need to say anything more than that, huh. 😅

6a9e8b2