Permission Denied fairseq \ examples
sriprad opened this issue · comments
Hi,
Thank you for the great library. Very useful. I tried to install in my machine and it landed up in error. Can you please let me know if i am missing anything,?
ERROR: Command errored out with exit status 1:
command: 'C:\Program Files\Anaconda\python.exe' 'C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py' get_requires_for_build_wheel 'C:\Users\Public\Documents\Wondershare\CreatorTemp\tmp90q37bss'
cwd: C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-zxfcubdl\fairseq
Complete output (31 lines):
Traceback (most recent call last):
File "setup.py", line 214, in
do_setup(package_data)
File "setup.py", line 136, in do_setup
setup(
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools_init_.py", line 152, in setup
install_setup_requires(attrs)
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools_init.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 60, in fetch_build_eggs
raise SetupRequirementsError(specifier_list)
setuptools.build_meta.SetupRequirementsError: ['cython', 'numpy', 'setuptools>=18.0']
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py", line 280, in
main()
File "C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py", line 114, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 149, in get_requires_for_build_wheel
return self._get_build_requires(
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 130, in _get_build_requires
self.run_setup()
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 145, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 217, in
os.unlink(fairseq_examples)
PermissionError: [WinError 5] Access is denied: 'fairseq\examples'
ERROR: Command errored out with exit status 1: 'C:\Program Files\Anaconda\python.exe' 'C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py' get_requires_for_build_wheel 'C:\Users\Public\Documents\Wondershare\CreatorTemp\tmp90q37bss' Check the logs for full command output.
@sriprad
The issue is with fairseq, it is sadly not compatible with windows due to a badly written setup script.
Fairseq is requried for the mBART and m2m model.
If you use opus-mt, fairseq is not required. You can install it then like this:
pip install --no-deps easynmt
pip install tqdm transformers numpy nltk sentencepiece
You also need pytorch:
https://pytorch.org/get-started/locally/
If you need automatic language detection, you also need fastText, which can be installed like this:
pip install fasttext
or when you use Anaconda: https://anaconda.org/conda-forge/fasttext
In that case you can use the opus-mt model.
Thank you so much @nreimers . Really helpful. However i landed up in another issue may be not related to library but got this one if you can help. I always landed up in the wrong place to download this. if you can help me here.
building 'fasttext_pybind' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
Also i am running the code like this hope this is correct.
from easynmt import EasyNMT
model = EasyNMT('opus-mt')
#Translate a single sentence to German
print(model.translate('This is a sentence we want to translate to German', target_lang='de'))
#Translate several sentences to German
sentences = ['You can define a list with sentences.',
'All sentences are translated to your target language.',
'Note, you could also mix the languages of the sentences.']
print(model.translate(sentences, target_lang='de'))
Thank you so much @nreimers . Really helpful. However i landed up in another issue may be not related to library but got this one if you can help. I always landed up in the wrong place to download this. if you can help me here.
building 'fasttext_pybind' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
fastText is used for the automatic language detection.
When on Windows and using Anaconda, you can install it like this:
https://anaconda.org/conda-forge/fasttext
Yes, the code you showed is correct.
If you provide the source language (source_lang=), fastText is not needed. In that case, you don't have to install it.
Translation of documents (like word, powerpoint, excel) is sadly not yet supported (it is on the agenda, help appreciated).
But you could implement it yourself for excel:
- Store it as CSV
- Load it with Python
- Translate all entries
- Write it back to CSV
- Load with excel
Thank you @nreimers if i provide (source_lang=) then how will be the structure of the code like what changes i need to make below to include
- source_lang =
- pass the csv file.
from easynmt import EasyNMT
model = EasyNMT('opus-mt')
#Translate a single sentence to German
print(model.translate('This is a sentence we want to translate to German', target_lang='de'))
#Translate several sentences to German
sentences = ['You can define a list with sentences.',
'All sentences are translated to your target language.',
'Note, you could also mix the languages of the sentences.']
print(model.translate(sentences, target_lang='de'))
Sure happy to support in development
You code must then look like this:
from easynmt import EasyNMT
model = EasyNMT('opus-mt')
#Translate a single sentence to German
print(model.translate('This is a sentence we want to translate to German', source_lang='en', target_lang='de'))
Note, with version 1.0.2:
https://github.com/UKPLab/EasyNMT/releases/tag/v1.0.2
I added two more options for language detection that are compatible with Windows:
pip install langid
If fastText is not installed, it will fall back to either langid or to langdetect.
Thank you @nreimers . I could run it with your examples.
Please find below i am trying to upload a csv with a column needs to be translated.
how should i pass the dataframe?
thanks
df = pd.read_csv("C:/xx/trans.csv",encoding = 'unicode_escape')
#how to pass the dataframe?
print(model.translate(, source_lang='en', target_lang='de'))
Hi @sriprad
You could try:
df['translated_text'] = model.translate(df['source_text'], source_lang='en', target_lang='de')
Not sure how the pandas DF looks for your csv. You might have to update the column names in the Python code.
Thank you @nreimers . very helpful.
Do you need GPU to run it? I am running the translation on 10k rows of data. With each row has varying line length of 5 to 8. It's been running for last 30 mins and still running.
It has finally run with 1 hour 25 mins. But brilliant . The translation is amazing. Great work @nreimers
Happy to hear that :)
You can pass the show_progress_bar=True to the fit method.
But yes, these models are quite slow on a CPU (see Readme). I can recommend to use Google Colab, there, you get a GPU for free which significantly speeds up the processing.
Thank you @nreimers :). sure i will check in with Google Colab. But not sure if we are allowed to use Google Colab for official purpose.
@sriprad
A docker container will soon be published. This will make it easy to run it (as long as you have docker installed).
@nreimers thank you . Is there a possibility of extending this to convert word doc? or lengthy contracts please?
@sriprad
Translating word docs is quite difficult, as docx is quite a complex format. Further, the content is mixed together with style & format commands. So extracting the text, translating it, and putting it back to a valid a nicely formatted word document is non-trivial.
This is also happening to me in Arch Linux.
Edit: Fixed it using Python 3.7 instead of Python 3.9
@nreimers If you can somehow use Okapi filters, you would be able to use most formats.
https://okapiframework.org/wiki/index.php/Filters
Or maybe, give higher priority to xliff files, since this is the main format in translation industry.
Im having trouble trying to download easyNMT,, when i do 'pip install easynmt' i get this error:
Building wheels for collected packages: fasttext
Building wheel for fasttext (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [75 lines of output]
...
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for fasttext
Running setup.py clean for fasttext
Failed to build fasttext
ERROR: Could not build wheels for fasttext, which is required to install pyproject.toml-based projects.
It also says that fasttext is required for easynmt 2.0.2