NER for species is no longer working in kazu 1.3.0
Longdexin opened this issue · comments
Hi, this is a great project, nice work. I have updated my kazu version to 1.3.0 by building a new docker image, but when I tried it on the swagger docs page, not everything was fine. I got some error message like below:
[
{
"idx": "154cfa347b484204904fdec07b5ab3cb",
"sections": [
{
"text": "There is a black mouse in the room.",
"name": "na"
}
],
"metadata": {
"PROCESSING_EXCEPTION": "batch failed: affected ids: ['154cfa347b484204904fdec07b5ab3cb']\nTraceback (most recent call last):\n File \"/home/ray/kazu/steps/step.py\", line 140, in step_call\n batch_doc_callable(self, docs)\n File \"/home/ray/kazu/steps/ner/hf_token_classification.py\", line 98, in __call__\n activations = self.get_activations(loader)\n File \"/home/ray/kazu/steps/ner/hf_token_classification.py\", line 121, in get_activations\n [\n File \"/home/ray/kazu/steps/ner/hf_token_classification.py\", line 122, in <listcomp>\n x.logits # type: ignore[union-attr]\nAttributeError: 'dict' object has no attribute 'logits'\n"
}
}
]In my current work, I have to tag some species words in the text. I noticed it said 'dict' object has no attribute 'logits' in the last line. Does it have something to do with the misbehaviour of species NER? How to fix it? How to make the species NER work again? Please offer some help, thanks a lot.
Thanks for trying it out!
Short description of the issue and KAZU's path to fixing it 'properly'
This is an issue with pytorch-lightning (one of the dependencies of kazu). We've fixed this internally, but haven't open sourced yet. We have a 1.3.2 release ready that pins dependencies as needed on our end, and a 1.4.0 release that actually removes PyTorch-lightning as a top-level dependency, because of these kinds of issues, and the fact that it only really relates to our model training code. Let me see if there's anything stopping us doing that, I should get back to you shortly.
Workaround
In the meantime, you can work around this by specifying a dependency in your project on:
lightning-utilities==0.9.0
which will resolve the problem.
Full explanation
We specify a dependency on pytorch-lightning>=1.7.4,<2.0.0, which has as a dependency lightning-utilities, produced by the same team. The PyTorch Lightning team released a new 0.10.0 version lightning-utilities which broke older releases of pytorch-lightning, and their own version pinning was incorrect in that it didn't prevent an inappropriate version of lightning-utilities from being installed. 'Manually' specifying lightning-utilities==0.9.0 therefore fixes the issue (without upgrading the pytorch-lightning version, which causes problems for us).
Hi,
I've just open sourced our (previously internal) v1.3.2 and v1.4.0 releases. Both of these releases have resolved this problem. Sorry for not getting these out sooner, and thanks for reporting the issue.
I'm closing the issue - if you're still seeing the issue please re-open though. And if you see other problems, please feel free to open new issues!