add native Windows support
hanxiao opened this issue · comments
Describe the feature
Jina can not run on Windows natively, please add your issues/observations here. We need community help on this one.
There are several knowns issues,
uvloop
does not support Windows.uvloop
is used in jina by default for boosting the performances, but it is known in the uvloop community. MagicStack/uvloop#14.SystemLogHandler
uses TCP on unix while UDP on Windows.- Multiprocessing in Windows does not work for jina.
Traceback (most recent call last):
File "C:\Users\magic\AppData\Local\Programs\Python\Python38-32\Scripts\jina-script.py", line 33, in <module>
sys.exit(load_entry_point('jina', 'console_scripts', 'jina')())
File "c:\users\magic\codes\github\jina-master\cli\__init__.py", line 90, in main
getattr(api, args.cli.replace('-', '_'))(args)
File "c:\users\magic\codes\github\jina-master\cli\api.py", line 8, in pod
with Pod(args) as p:
File "c:\users\magic\codes\github\jina-master\jina\peapods\pod.py", line 254, in __enter__
return self.start()
File "c:\users\magic\codes\github\jina-master\jina\peapods\pod.py", line 224, in start
self.enter_context(p)
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\contextlib.py", line 425, in enter_context
result = _cm_type.__enter__(cm)
File "c:\users\magic\codes\github\jina-master\jina\peapods\pea.py", line 424, in __enter__
return self.start()
File "c:\users\magic\codes\github\jina-master\jina\peapods\pea.py", line 398, in start
super().start()
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'jina.peapods.pea.BasePea'>: it's not the same object as jina.peapods.pea.BasePea
PS C:\Users\magic\Codes\github\tmp> Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\spawn.py", line 102, in spawn_main
source_process = _winapi.OpenProcess(
OSError: [WinError 87]
Your proposal
-
uvloop
is installed by default when runningpip install "jina[core]"
. One needs to figure out a way to avoid installuvloop
in theextra-requirements.txt
-
Add the logic for processing windows.
https://github.com/jina-ai/jina/blob/master/jina/logging/logger.py#L155
https://github.com/jina-ai/jina/blob/master/jina/logging/logger.py#L164 -
Need more time to look into the issue.
Environment
jina 0.7.5
jina-proto 0.0.75
jina-vcs-tag (unset)
libzmq 4.3.2
pyzmq 1.18.1
protobuf 3.13.0
proto-backend cpp
grpcio 1.27.2
ruamel.yaml 0.16.10
python 3.7.5
platform Darwin
platform-release 19.5.0
platform-version Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64
architecture x86_64
processor i386
jina-resources /Users/nanwang/Codes/jina-ai/jina/jina/resources
JINA_ARRAY_QUANT (unset)
JINA_BINARY_DELIMITER (unset)
JINA_CONTRIB_MODULE (unset)
JINA_CONTRIB_MODULE_IS_LOADING(unset)
JINA_CONTROL_PORT (unset)
JINA_DB_COLLECTION (unset)
JINA_DB_HOSTNAME (unset)
JINA_DB_NAME (unset)
JINA_DB_PASSWORD (unset)
JINA_DB_USERNAME (unset)
JINA_DEFAULT_HOST (unset)
JINA_DISABLE_UVLOOP (unset)
JINA_EXECUTOR_WORKDIR (unset)
JINA_FULL_CLI (unset)
JINA_IPC_SOCK_TMP (unset)
JINA_LOG_CONFIG (unset)
JINA_LOG_NO_COLOR (unset)
JINA_POD_NAME (unset)
JINA_PROFILING (unset)
JINA_RANDOM_PORTS (unset)
JINA_SOCKET_HWM (unset)
JINA_TEST_GPU (unset)
JINA_TEST_PRETRAINED (unset)
JINA_VCS_VERSION (unset)
JINA_WARN_UNNAMED (unset)
Screenshots
@nan-wang @deepankarm please post your Windows running errors here
@nan-wang @deepankarm please post your Windows running errors here
Updated the issue accordingly.
This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 4 days
RuntimeError: uvloop does not support Windows at the moment
@yogeshhk hi, there. Thanks for your interests. The native Windows support is not there yet. You can try Jina with WSL. https://docs.jina.ai/chapters/install/os/on-wsl.html