FileNotFoundError: [Errno 2] for local server POST

Question

FileNotFoundError: [Errno 2] for local server POST

starmpcc opened this issue 2 years ago · comments

Hello. I installed BERN2 in my VM (Ubuntu 18.04 x64, 1 GPU).
It succesfully installed, however, if I run example post script on READAME.md, then the below error occured.
(Server console log)

[23/Mar/2022 06:02:40.831728] [550f7242b14dc039a8a5ee6aa9233f3472dd7b4a33484f5955175a4d] GNormPlus 0.0001380443572998047 sec
Traceback (most recent call last):
  File "/home/vessl/BERN2/bern2/bern2.py", line 106, in annotate_text
    output = self.tag_entities(text, base_name)
  File "/home/vessl/BERN2/bern2/bern2.py", line 358, in tag_entities
    async_result = loop.run_until_complete(self.async_ner(arguments_for_coroutines))
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/home/vessl/BERN2/bern2/bern2.py", line 491, in async_ner
    result = await asyncio.gather(*coroutines)
  File "/home/vessl/BERN2/bern2/bern2.py", line 533, in _ner_wrap
    with open(output_mtner, 'r', encoding='utf-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: './multi_ner/output/550f7242b14dc039a8a5ee6aa9233f3472dd7b4a33484f5955175a4d.PubTator.json'

[2022-03-23 06:02:41,425] ERROR in app: Exception on /plain [POST]
Traceback (most recent call last):
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/vessl/BERN2/app/__init__.py", line 85, in plain_api
    return Response(json.dumps({"error_message": result_dict["error_message"]}), status=404, content_type='application/json')
NameError: name 'Response' is not defined
127.0.0.1 - - [23/Mar/2022 06:02:41] "POST /plain HTTP/1.1" 500 -

nohup_multi_ner.out log

MTNER init_t 28.547 sec.
1it [00:00, 185.89it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  2.28it/s]
Traceback (most recent call last):
  File "multi_ner/ner_server.py", line 91, in <module>
    run_server(mt_ner, args)
  File "multi_ner/ner_server.py", line 63, in run_server
    mtner_recognize(model, dict_path, base_name, args)
  File "multi_ner/ner_server.py", line 47, in mtner_recognize
    with open(output_mt_ner, 'w', encoding='utf-8') as f:
PermissionError: [Errno 13] Permission denied: 'multi_ner/output/550f7242b14dc039a8a5ee6aa9233f3472dd7b4a33484f5955175a4d.PubTator.json'

Thank you.

Mujeen Sung · Answer 1 · Wed Mar 23 2022 15:41:19 GMT+0800 (China Standard Time)

Hi, @starmpcc

Thanks for reaching out to us.

According to the error log, it seems like the output folder needs the write permission.

PermissionError: [Errno 13] Permission denied: 'multi_ner/output/550f7242b14dc039a8a5ee6aa9233f3472dd7b4a33484f5955175a4d.PubTator.json'

I think there are two options.

Executing sudo ls and restarting BERN2.
Giving the write permission to the folder by executing chmod 755 multi_ner/output.

I hope this answer helps you.

Junu Kim · Answer 2 · Wed Mar 23 2022 16:16:59 GMT+0800 (China Standard Time)

Thank you!
As you expected, some of directories set as root permission so i executed chown -R USERNAME . and chgrp -R USERNAME .
However, the execution was failure with similiar error.

[23/Mar/2022 08:13:19.050116] [55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7] Multi-task NER 0.05367755889892578 sec, #entities: 2
Traceback (most recent call last):
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/shutil.py", line 566, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator' -> './resources/tmVarJava/input/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator.PubTator.Gene'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vessl/BERN2/bern2/bern2.py", line 106, in annotate_text
    output = self.tag_entities(text, base_name)
  File "/home/vessl/BERN2/bern2/bern2.py", line 371, in tag_entities
    shutil.move(output_gnormplus, input_tmvar_gene)
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator'

[2022-03-23 08:13:19,052] ERROR in app: Exception on /plain [POST]
Traceback (most recent call last):
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/vessl/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/vessl/BERN2/app/__init__.py", line 85, in plain_api
    return Response(json.dumps({"error_message": result_dict["error_message"]}), status=404, content_type='application/json')
NameError: name 'Response' is not defined

Moreover, after failure and kill server, some of directories were changed to root. permission. I already checked all files and directories had user permission before execusion.
More precisely, the owner of multi_ner/output was changed from user to root.

(bern2) vessl@workspace-5fhfe79px43n-0:~/BERN2$ chown -R vessl .
chown: changing ownership of './resources/GNormPlusJava/input/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator': Operation not permitted
chown: changing ownership of './resources/tmVarJava/input/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator': Operation not permitted
chown: changing ownership of './multi_ner/output/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator.json': Operation not permitted
chown: changing ownership of './multi_ner/input/55a389534f6fe973b440b0c5bd0821aae7f7d339423b4cc486e1e0d7.PubTator.PubTator': Operation not permitted

I guess sudo in run_bern2.sh might caused the problem. However, delete that keyword didn't make change.

ixalidamus · Answer 3 · Tue Mar 29 2022 17:35:44 GMT+0800 (China Standard Time)

similar error

File "/root/BERN2/app/result_parser.py", line 210, in parse_resultsorg_text = result_dict['text'] KeyError: 'text' [29/Mar/2022 09:28:50] "POST /senddata HTTP/1.1" 500 -

lengluodefanhua · Answer 4 · Thu Apr 07 2022 23:21:44 GMT+0800 (China Standard Time)

similar error...

Mujeen Sung · Answer 5 · Fri Apr 08 2022 13:50:38 GMT+0800 (China Standard Time)

Hi @starmpcc @ixalidamus @lengluodefanhua

First of all, I removed sudo from run_bern2.sh after verifying that BERN2 can run without it.

It's most likely that the external modules (Multi-NER, tmVar, GNormPlus, etc.) are causing the issue. The reasons are various, so we need to check the logs.

The log files are saved in /logs so you can check them to see which module is causing the issue.

If you find anything suspicious in the logs, please share it with me so that I can help you.

lengluodefanhua · Answer 6 · Fri Apr 08 2022 17:09:37 GMT+0800 (China Standard Time)

Hi @starmpcc @ixalidamus @lengluodefanhua

First of all, I removed sudo from run_bern2.sh after verifying that BERN2 can run without it.

It's most likely that the external modules (Multi-NER, tmVar, GNormPlus, etc.) are causing the issue. The reasons are various, so we need to check the logs.

The log files are saved in /logs so you can check them to see which module is causing the issue.

If you find anything suspicious in the logs, please share it with me so that I can help you.

The logs is below:

[Error] : CRF++ is loading fault. Please follow below instruction to re-install CRF++.
Execute "Installation.sh" to re-compile crf_test first.

$ sh Installation.sh

If failed by using Installation.sh, please reinstall CRF++ by below steps:

Download CRF++-0.58.tar.gz from https://drive.google.com/folderview?id=0B4y35FiV1wh7fngteFhHQUN2Y1B5eUJBNHZUemJYQV9VWlBUb3JlX0xBdWVZTWtSbVBneU0&usp=drive_web#list
Uncompress the file by

$ tar -zxvf CRF++-0.58.tar.gz
Move all files and folders under CRF++-0.58 to the [CRF] folder.
Execute below instructions in CRF folder.
$ ./configure
$ make
$ su
$ make install

I've tried all of the following methods. It still doesn't work.

Mujeen Sung · Answer 7 · Fri Apr 08 2022 20:19:38 GMT+0800 (China Standard Time)

@lengluodefanhua

Thanks for sharing.

After installing CRF++ following the steps, do you still see the same error logs? Did you happen to come across any errors while installing the CRF++ module? Are you using Linux or Windows?

shi4712shi · Answer 8 · Mon Apr 11 2022 14:26:34 GMT+0800 (China Standard Time)

@lengluodefanhua

Thanks for sharing.

After installing CRF++ following the steps, do you still see the same error logs? Did you happen to come across any errors while installing the CRF++ module? Are you using Linux or Windows?

Hi, @mjeensung:
I removed the resources directory and re-installed CRF++. And there is my situation:

After installing CRF++ following the steps, do you still see the same error logs?

Yes, following the steps, the error logs are the same as before.

Did you happen to come across any errors while installing the CRF++ module?

There is no error, except for a warning:
libtool: install: warning: remember to run libtool --finish /usr/local/lib'`.
And it's no help to run this command for this error.

Are you using Linux or Windows?

I am using Linux.

Minji Kim · Answer 9 · Wed Apr 20 2022 13:21:43 GMT+0800 (China Standard Time)

Hi, I am getting the same error- the logs for [nohup_bern2.out] show:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/mjkim/BERN2/bern2/bern2.py", line 106, in annotate_text
    output = self.tag_entities(text, base_name)
  File "/data/mjkim/BERN2/bern2/bern2.py", line 371, in tag_entities
    shutil.move(output_gnormplus, input_tmvar_gene)
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 571, in move
    copy_function(src, real_dst)
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 257, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/6df3ec056d96a405b8e2f77748f77716f089292e77672c761be974d6.PubTator'

and for [nohup_gnormplus.out] shows:

Starting GNormPlus Service at 127.0.1.1:18895
Loading Gene Dictionary : Processing ... 
Loading Gene Dictionary : Processing Time:9.087sec
Ready
java.io.IOException: Cannot run program "./CRF/crf_test": error=2, No such file or directory
Starting GNormPlus Service at 127.0.1.1:18895
Loading Gene Dictionary : Processing ... 
Loading Gene Dictionary : Processing Time:9.61sec
Ready
[Error] : CRF++ is loading fault. Please follow below instruction to re-install CRF++.
Execute "Installation.sh" to re-compile crf_test first.

	$ sh Installation.sh

If failed by using Installation.sh, please reinstall CRF++ by below steps:

1) Download CRF++-0.58.tar.gz from https://drive.google.com/folderview?id=0B4y35FiV1wh7fngteFhHQUN2Y1B5eUJBNHZUemJYQV9VWlBUb3JlX0xBdWVZTWtSbVBneU0&usp=drive_web#list
2) Uncompress the file by 

	$ tar -zxvf CRF++-0.58.tar.gz

3) Move all files and folders under CRF++-0.58 to the [CRF] folder.

4) Execute below instructions in CRF folder.
	$ ./configure 
	$ make
	$ su
	$ make install

I've tried removing and redownloading the resources file, and also have tried to reinstall CRF++ using the steps listed above. I am on linux, have also downloaded CRF as provided in the Readme file. How can I fix this error?

Mujeen Sung · Answer 10 · Wed Apr 20 2022 15:03:51 GMT+0800 (China Standard Time)

Hi @minjiijnim

Have you tried installing CRF++ following the steps on the readme?

# install CRF (Only for Linux/MacOS)
cd resources/GNormPlusJava/CRF
./configure --prefix="$HOME"
make
make install
cd ../../..

Can you find crf_test in ./resources/GNormPlusJava/CRF?

Other than that, what is the version of your C++ compiler?
According to https://taku910.github.io/crfpp/, C++ compiler (gcc 3.0 or higher) is required to install CRF++.

Minji Kim · Answer 11 · Wed Apr 20 2022 15:13:36 GMT+0800 (China Standard Time)

Hi @minjiijnim

Have you tried installing CRF++ following the steps on the readme?
# install CRF (Only for Linux/MacOS)
cd resources/GNormPlusJava/CRF
./configure --prefix="$HOME"
make
make install
cd ../../..
Can you find crf_test in ./resources/GNormPlusJava/CRF?

Other than that, what is the version of your C++ compiler? According to https://taku910.github.io/crfpp/, C++ compiler (gcc 3.0 or higher) is required to install CRF++.

Hi! Thanks for the reply- I just checked the C++ complier and it is 5.4.0
Also, yes I have already installed CRF following the steps on Readme.
I also found 'crf test' in the path you mentioned.

Also, from running the program again, I noticed this:

[Errno 111] Connection refused
[20/Apr/2022 17:03:10.283182] [ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c] tmVar 2.0 0.0004878044128417969 sec
[20/Apr/2022 17:03:10.649087] [ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c] Multi-task NER 0.36307597160339355 sec, #entities: 2
[20/Apr/2022 17:03:11.132458] [ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c] GNormPlus 0.8491368293762207 sec
Traceback (most recent call last):
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 557, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c.PubTator' -> './resources/tmVarJava/input/ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c.PubTator.PubTator.Gene'

I checked the tmVarJava/input file name and it don't seem to match with the name ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c.PubTator.PubTator.Gene but instead just has ba1482656d834f54ed08e197d6e59c6fbc65897952d38a5af448876c.PubTator (without the .PubTator.Gene at the end)

Mujeen Sung · Answer 12 · Fri Apr 29 2022 13:28:58 GMT+0800 (China Standard Time)

Hi @minjiijnim @lengluodefanhua @shi4712shi @starmpcc @ixalidamus

Greatly thanks for your patience.
I was finally able to reproduce the issue.

This issue is caused by the old crf_test file.
This file should be newly compiled depending on your server environments, which can be done by calling make install.
But, I found that when the file is already existing in the foler, compiling it is ignored.

So, the workaround for this issue is as follows:

Remove resources/GNormPlusJava/CRF
Download CRF++-0.58.tar.gz from https://drive.google.com/folderview?id=0B4y35FiV1wh7fngteFhHQUN2Y1B5eUJBNHZUemJYQV9VWlBUb3JlX0xBdWVZTWtSbVBneU0&usp=drive_web#list
Uncompress the file by
$ tar -zxvf CRF++-0.58.tar.gz
Rename the folder [CRF++-0.58] to [CRF].
Execute below instructions in CRF folder.
$ ./configure --prefix="$HOME"
$ make
$ make install

Minji Kim · Answer 13 · Fri Apr 29 2022 15:38:20 GMT+0800 (China Standard Time)

@mjeensung

Thank you so much for the solution- the GNormPlusJava resource error seems to be fixed.
However, now there seems to be another error- this time with tmVarJava :'-)

Here is the error:

[29/Apr/2022 16:09:22.031069] id: 94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90
[Errno 111] Connection refused
[29/Apr/2022 16:09:22.033594] [94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90] tmVar 2.0 0.0004341602325439453 sec
[29/Apr/2022 16:09:22.433258] [94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90] GNormPlus 0.39952659606933594 sec
[29/Apr/2022 16:09:22.434386] [94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90] Multi-task NER 0.398362398147583 sec, #entities: 2
Traceback (most recent call last):
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 557, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: './resources/tmVarJava/output/94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90.PubTator.PubTator' -> './resources/tmVarJava/input/94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90.PubTator.PubTator'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/mjkim/BERN2/bern2/bern2.py", line 106, in annotate_text
    output = self.tag_entities(text, base_name)
  File "/data/mjkim/BERN2/bern2/bern2.py", line 372, in tag_entities
    shutil.move(output_tmvar_ner, input_tmvar_norm)
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 571, in move
    copy_function(src, real_dst)
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 257, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/data/mjkim/anaconda3/envs/bern/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './resources/tmVarJava/output/94a2292b0aef74e2f32547207dd43d4f8f4ce65aba32f1fa0ab96d90.PubTator.PubTator'

Also, nohup_tmvar.out reads:

Starting tmVar 2.0 Service at 127.0.1.1:18896
Exception in thread "main" java.lang.NoClassDefFoundError: edu/stanford/nlp/tagger/maxent/MaxentTagger
	at kr.ac.korea.dmis.tmVar2.<init>(tmVar2.java:66)
	at kr.ac.korea.dmis.tmVar2Server.<init>(tmVar2Server.java:23)
	at kr.ac.korea.dmis.tmVar2Server.main(tmVar2Server.java:89)
Caused by: java.lang.ClassNotFoundException: edu.stanford.nlp.tagger.maxent.MaxentTagger
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 3 more

(I've followed the steps for downloading and executing the new CRF folder as advised for resources/tmVarJava as well, but it doesn't seem to change the error.)

Mujeen Sung · Answer 14 · Fri Apr 29 2022 16:39:53 GMT+0800 (China Standard Time)

@minjiijnim

Thanks for sharing!

Can you check if there is this file resources/tmVarJava/stanford-corenlp-3.5.2.jar?

If you find it, could you compare the md5 checksum of it to mine?

$ md5sum stanford-corenlp-3.5.2.jar
>> c9827171f5badc07f475a66b2768c095 stanford-corenlp-3.5.2.jar

Minji Kim · Answer 15 · Fri Apr 29 2022 16:51:04 GMT+0800 (China Standard Time)

@mjeensung

I don't seem to have the file at all :'(
Here is the list of files in my resources/tmVarJava:

(bern) mjkim@ubuntu:~/BERN2/resources/tmVarJava$ ls
COPYING     input                   sqlite-jdbc-3.8.11.2.jar
corpus      Installation.sh         src
CRF         output.txt              tmVar2Server_20220112.main.jar
Database    readme.txt              tmVar2Server.main.jar
DISCLAIMER  sqlite-jdbc-3.20.0.jar  tmVar.sh

Mujeen Sung · Answer 16 · Fri Apr 29 2022 17:06:00 GMT+0800 (China Standard Time)

@minjiijnim

It is really odd, but hopefully it can be easily solved by adding the file.
Could you download the following file and move it to /BERN2/resources/tmVarJava?

wget http://nlp.dmis.korea.edu/projects/bern2-sung-et-al-2022/stanford-corenlp-3.5.2.jar

Mujeen Sung · Answer 17 · Mon May 02 2022 09:09:46 GMT+0800 (China Standard Time)

@minjiijnim

Sorry for the wrong url. I've updated the url above.

Minji Kim · Answer 18 · Tue May 03 2022 13:12:33 GMT+0800 (China Standard Time)

@mjeensung

It seems to be working good now, thank you!! :)

Cavalry With Politeness · Answer 19 · Tue May 03 2022 16:50:51 GMT+0800 (China Standard Time)

Hi, I'm getting a similiar error.

But I tried to reinstalled CRF++-0.58 ,renamed it and tried:
$ ./configure --prefix="$HOME"
$ make
$ make install

the log still shows:

Arguments:
	host: 0.0.0.0
	port: 8888
	mtner_home: ./multi_ner
	mtner_port: 18894
	gnormplus_home: ./resources/GNormPlusJava
	gnormplus_port: 18895
	tmvar2_home: ./resources/tmVarJava
	tmvar2_port: 18896
	cache_host: localhost
	cache_port: 27017
	gene_norm_port: 18888
	disease_norm_port: 18892
	use_neural_normalizer: True
	keep_files: False
	front_dev: False
/home/scholar1/bern2/BERN2/app
[03/May/2022 16:26:11.494623] BERN2 LOADING..
./output 0
./resources/GNormPlusJava/input 1
./resources/tmVarJava/input 1
./multi_ner/input 1
./multi_ner/tmp 0
./multi_ner/output 1
start loading neural normalizer..
neural_disease_normalizer is loaded.. model=dmis-lab/biosyn-sapbert-bc5cdr-disease , dictionary=resources/normalization/normalizers/neural_norm_caches/dict_Disease_20210630.txt.pk
neural_chemical_normalizer is loaded.. model=dmis-lab/biosyn-sapbert-bc5cdr-chemical , dictionary=resources/normalization/normalizers/neural_norm_caches/dict_ChemicalCompound_20210630.txt.pk
neural_gene_normalizer is loaded.. model=dmis-lab/biosyn-sapbert-bc2gn , dictionary=resources/normalization/normalizers/neural_norm_caches/dict_Gene.txt.pk
[03/May/2022 16:31:20.109368] BERN2 LOADED..
 * Serving Flask app 'app' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses (0.0.0.0)
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://127.0.0.1:8888
 * Running on http://192.168.1.100:8888 (Press CTRL+C to quit)
[03/May/2022 16:31:45.433902] id: 618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39
[Errno 111] Connection refused
[03/May/2022 16:31:45.667640] [618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39] GNormPlus 0.0005099773406982422 sec
[03/May/2022 16:31:58.768150] [618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39] Multi-task NER 13.100327014923096 sec, #entities: 2
[03/May/2022 16:32:03.884995] [618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39] tmVar 2.0 18.344019412994385 sec
Traceback (most recent call last):
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 566, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39.PubTator' -> './resources/tmVarJava/input/618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39.PubTator.PubTator.Gene'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/scholar1/bern2/BERN2/bern2/bern2.py", line 106, in annotate_text
    output = self.tag_entities(text, base_name)
  File "/home/scholar1/bern2/BERN2/bern2/bern2.py", line 371, in tag_entities
    shutil.move(output_gnormplus, input_tmvar_gene)
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/618c194f757312b3ded0db7102f2bfaa49de13599c0136c3ad50fe39.PubTator'

127.0.0.1 - - [03/May/2022 16:32:04] "�[33mPOST /plain HTTP/1.1�[0m" 404 -

In addition, I am on Linux and my gcc version is 7.5.0(Ubuntu 7.5.0-3ubuntu1~18.04) .i notice the information above, crf_test is in my GNormPlusJava/CRF directory and there is nothing in my GNormPlusJava/output directory but I don't know how to fix it.

Mujeen Sung · Answer 20 · Tue May 03 2022 17:08:58 GMT+0800 (China Standard Time)

Hi @yuetieqi-meow

Thanks for reaching out to us.

I'd like to make sure that you removed the resources/GNormPlusJava/CRF folder before reinstalling it.
Also, could you check the logs in the logs folder and see which module is causing the issue?

Cavalry With Politeness · Answer 21 · Tue May 03 2022 18:20:24 GMT+0800 (China Standard Time)

Hi @yuetieqi-meow

Thanks for reaching out to us.

I'd like to make sure that you removed the resources/GNormPlusJava/CRF folder before reinstalling it. Also, could you check the logs in the logs folder and see which module is causing the issue?

Thanks for replying!
I felt puzzled for the logs.The log 'nohup_bern2.out' shows:

Arguments:
	host: 0.0.0.0
	port: 8888
	mtner_home: ./multi_ner
	mtner_port: 18894
	gnormplus_home: ./resources/GNormPlusJava
	gnormplus_port: 18895
	tmvar2_home: ./resources/tmVarJava
	tmvar2_port: 18896
	cache_host: localhost
	cache_port: 27017
	gene_norm_port: 18888
	disease_norm_port: 18892
	use_neural_normalizer: True
	keep_files: False
	front_dev: False
/home/scholar1/bern2/BERN2/app
[03/May/2022 17:44:45.867291] BERN2 LOADING..
./output 0
./resources/GNormPlusJava/input 1
./resources/tmVarJava/input 1
./multi_ner/input 1
./multi_ner/tmp 0
./multi_ner/output 1
start loading neural normalizer..
neural_disease_normalizer is loaded.. model=dmis-lab/biosyn-sapbert-bc5cdr-disease , dictionary=resources/normalization/normalizers/neural_norm_caches/dict_Disease_20210630.txt.pk
neural_chemical_normalizer is loaded.. model=dmis-lab/biosyn-sapbert-bc5cdr-chemical , dictionary=resources/normalization/normalizers/neural_norm_caches/dict_ChemicalCompound_20210630.txt.pk
neural_gene_normalizer is loaded.. model=dmis-lab/biosyn-sapbert-bc2gn , dictionary=resources/normalization/normalizers/neural_norm_caches/dict_Gene.txt.pk
[03/May/2022 17:51:05.639551] BERN2 LOADED..
 * Serving Flask app 'app' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses (0.0.0.0)
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://127.0.0.1:8888
 * Running on http://192.168.1.100:8888 (Press CTRL+C to quit)
[03/May/2022 17:51:23.497879] id: 2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe
[Errno 111] Connection refused
[03/May/2022 17:51:23.728513] [2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe] GNormPlus 0.0005197525024414062 sec
[03/May/2022 17:51:31.127499] [2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe] Multi-task NER 7.398807048797607 sec, #entities: 2
[03/May/2022 17:51:33.748486] [2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe] tmVar 2.0 10.134171485900879 sec
Traceback (most recent call last):
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 566, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe.PubTator' -> './resources/tmVarJava/input/2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe.PubTator.PubTator.Gene'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/scholar1/bern2/BERN2/bern2/bern2.py", line 106, in annotate_text
    output = self.tag_entities(text, base_name)
  File "/home/scholar1/bern2/BERN2/bern2/bern2.py", line 371, in tag_entities
    shutil.move(output_gnormplus, input_tmvar_gene)
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/scholar1/anaconda3/envs/bern2/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './resources/GNormPlusJava/output/2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe.PubTator'

127.0.0.1 - - [03/May/2022 17:51:33] "�[33mPOST /plain HTTP/1.1�[0m" 404 -

just like shown before.
However , I think all other 5 logs(I don't know whether there should be 5 logs) show the server goes all right:
nohup_disease_normalize.out:

Sieve loading .. 1504 ms, Ready

nohup_gene_normalize.out:

Ready (port 18888)

nohup_gnormplus.out:

Starting GNormPlus Service at 192.168.1.100:18895
Loading Gene Dictionary : Processing ... 
Loading Gene Dictionary : Processing Time:16.706sec
Ready

nohup_multi_ner.out:

MTNER init_t 101.298 sec.

0it [00:00, ?it/s]
1it [00:00, 19.95it/s]

Prediction:   0%|          | 0/1 [00:00<?, ?it/s]
Prediction: 100%|██████████| 1/1 [00:05<00:00,  5.75s/it]
Prediction: 100%|██████████| 1/1 [00:05<00:00,  5.76s/it]

nohup_tmvar.out:

Starting tmVar 2.0 Service at 192.168.1.100:18896
Reading POS tagger model from lib/taggers/english-left3words-distsim.tagger ... done [2.4 sec].
Loading tmVar : Processing Time:3.449sec
Ready
input/2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe.PubTator - (PubTator format) : Processing Time:5.732sec
ner success = 2549b3f9f01c6b899b9abd7d0092bf994ce8758974748003cba0dfbe.PubTator

And I'm sure that I removed the resources/GNormPlusJava/CRF.However, I retried it once again, and it doesn't work yet.

Mujeen Sung · Answer 22 · Thu Jun 02 2022 11:05:19 GMT+0800 (China Standard Time)

Hi @yuetieqi-meow

Sorry for the late response.

It's odd because I can't find any clue from your logs.
Does the issue still exist even after you kill and restart all of the processes?

Mujeen Sung · Answer 23 · Thu Jun 02 2022 11:54:07 GMT+0800 (China Standard Time)

Close this issue and follow up #24

Alexander Belikov · Answer 24 · Tue Jan 31 2023 06:37:40 GMT+0800 (China Standard Time)

Hi!
This issue is recurrent. After a few thousand iterations I encounter an error similar the one above

  File "/home/user/.pyenv/versions/3.7.13/lib/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/home/user/.pyenv/versions/3.7.13/lib/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/user/.pyenv/versions/3.7.13/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: './resources/tmVarJava/output/18e9560c7b11023>
[30/Jan/2023 22:51:26.035380] id: 4ecf1d2a9775caf06053c25253f869c26cf97c68a65a42f693ddbf5d
[Errno 111] Connection refused
[30/Jan/2023 22:51:26.036036] [4ecf1d2a9775caf06053c25253f869c26cf97c68a65a42f693ddbf5d] tmVar 2.0 0.>

In the logs in nohup_tmvar.out I find:

Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -1
	at java.base/java.lang.StringLatin1.charAt(StringLatin1.java:47)
	at java.base/java.lang.String.charAt(String.java:693)
	at java.base/java.lang.Character.codePointBefore(Character.java:8365)
	at java.base/java.util.regex.Pattern$CharPropertyGreedy.match(Pattern.java:4311)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$Start.match(Pattern.java:3619)
	at java.base/java.util.regex.Matcher.search(Matcher.java:1729)
	at java.base/java.util.regex.Matcher.find(Matcher.java:746)
	at java.base/java.util.regex.Matcher.replaceAll(Matcher.java:1178)
	at java.base/java.lang.String.replaceAll(String.java:2126)
	at tmVarlib.MentionRecognition.FeatureExtraction(MentionRecognition.java:782)
	at kr.ac.korea.dmis.tmVar2.tag_ner(tmVar2.java:262)
	at kr.ac.korea.dmis.tmVar2Server.run(tmVar2Server.java:48)
	at kr.ac.korea.dmis.tmVar2Server.<init>(tmVar2Server.java:29)
	at kr.ac.korea.dmis.tmVar2Server.main(tmVar2Server.java:89)

Alexander Belikov · Answer 25 · Tue Jan 31 2023 08:02:28 GMT+0800 (China Standard Time)

some investigation reveals that tmVar2Server dies sporadically (which is launched as)

nohup java -Xmx8G -Xms8G -jar tmVar2Server.main.jar 18896 >> ../../logs/nohup_tmvar.out 2>&1 &

restarting it fixes the pipeline

Alexander Belikov · Answer 26 · Tue Jan 31 2023 08:03:49 GMT+0800 (China Standard Time)

By the way,
tmVar2Server has been revamped as https://github.com/yuzhimanhua/Multi-BioNER.
Rewritten in python (?), which is a lot more transparent.

Or actually maybe it became https://github.com/ncbi/tmVar3 ?
The references are confusing.