jina-ai / jina

☁️ Build multimodal AI applications with cloud-native stack

Home Page:https://docs.jina.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

add native Windows support

hanxiao opened this issue · comments

Describe the feature

Jina can not run on Windows natively, please add your issues/observations here. We need community help on this one.

There are several knowns issues,

  1. uvloop does not support Windows. uvloop is used in jina by default for boosting the performances, but it is known in the uvloop community. MagicStack/uvloop#14.
  2. SystemLogHandler uses TCP on unix while UDP on Windows.
  3. Multiprocessing in Windows does not work for jina.
Traceback (most recent call last): 
  File "C:\Users\magic\AppData\Local\Programs\Python\Python38-32\Scripts\jina-script.py", line 33, in <module> 
    sys.exit(load_entry_point('jina', 'console_scripts', 'jina')()) 
  File "c:\users\magic\codes\github\jina-master\cli\__init__.py", line 90, in main 
    getattr(api, args.cli.replace('-', '_'))(args) 
  File "c:\users\magic\codes\github\jina-master\cli\api.py", line 8, in pod 
    with Pod(args) as p: 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pod.py", line 254, in __enter__ 
    return self.start() 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pod.py", line 224, in start 
    self.enter_context(p) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\contextlib.py", line 425, in enter_context 
    result = _cm_type.__enter__(cm) 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pea.py", line 424, in __enter__ 
    return self.start() 
  File "c:\users\magic\codes\github\jina-master\jina\peapods\pea.py", line 398, in start 
    super().start() 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\process.py", line 121, in start 
    self._popen = self._Popen(self) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\context.py", line 224, in _Popen 
    return _default_context.get_context().Process._Popen(process_obj) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\context.py", line 327, in _Popen 
    return Popen(process_obj) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__ 
    reduction.dump(process_obj, to_child) 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\reduction.py", line 60, in dump 
    ForkingPickler(file, protocol).dump(obj) 
_pickle.PicklingError: Can't pickle <class 'jina.peapods.pea.BasePea'>: it's not the same object as jina.peapods.pea.BasePea 
PS C:\Users\magic\Codes\github\tmp> Traceback (most recent call last): 
  File "<string>", line 1, in <module> 
  File "c:\users\magic\appdata\local\programs\python\python38-32\lib\multiprocessing\spawn.py", line 102, in spawn_main 
    source_process = _winapi.OpenProcess( 
OSError: [WinError 87] 

Your proposal

  1. uvloop is installed by default when running pip install "jina[core]". One needs to figure out a way to avoid install uvloop in the extra-requirements.txt

  2. Add the logic for processing windows.
    https://github.com/jina-ai/jina/blob/master/jina/logging/logger.py#L155
    https://github.com/jina-ai/jina/blob/master/jina/logging/logger.py#L164

  3. Need more time to look into the issue.


Environment

jina                          0.7.5
jina-proto                    0.0.75
jina-vcs-tag                  (unset)
libzmq                        4.3.2
pyzmq                         1.18.1
protobuf                      3.13.0
proto-backend                 cpp
grpcio                        1.27.2
ruamel.yaml                   0.16.10
python                        3.7.5
platform                      Darwin
platform-release              19.5.0
platform-version              Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64
architecture                  x86_64
processor                     i386
jina-resources                /Users/nanwang/Codes/jina-ai/jina/jina/resources
JINA_ARRAY_QUANT              (unset)
JINA_BINARY_DELIMITER         (unset)
JINA_CONTRIB_MODULE           (unset)
JINA_CONTRIB_MODULE_IS_LOADING(unset)
JINA_CONTROL_PORT             (unset)
JINA_DB_COLLECTION            (unset)
JINA_DB_HOSTNAME              (unset)
JINA_DB_NAME                  (unset)
JINA_DB_PASSWORD              (unset)
JINA_DB_USERNAME              (unset)
JINA_DEFAULT_HOST             (unset)
JINA_DISABLE_UVLOOP           (unset)
JINA_EXECUTOR_WORKDIR         (unset)
JINA_FULL_CLI                 (unset)
JINA_IPC_SOCK_TMP             (unset)
JINA_LOG_CONFIG               (unset)
JINA_LOG_NO_COLOR             (unset)
JINA_POD_NAME                 (unset)
JINA_PROFILING                (unset)
JINA_RANDOM_PORTS             (unset)
JINA_SOCKET_HWM               (unset)
JINA_TEST_GPU                 (unset)
JINA_TEST_PRETRAINED          (unset)
JINA_VCS_VERSION              (unset)
JINA_WARN_UNNAMED             (unset)

Screenshots

@nan-wang @deepankarm please post your Windows running errors here

@nan-wang @deepankarm please post your Windows running errors here

Updated the issue accordingly.

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 4 days

RuntimeError: uvloop does not support Windows at the moment

@yogeshhk hi, there. Thanks for your interests. The native Windows support is not there yet. You can try Jina with WSL. https://docs.jina.ai/chapters/install/os/on-wsl.html