[DOC] Better example of dask.neighbors in README
wuxianliang opened this issue · comments
Report incorrect documentation
Location of incorrect documentation
In README, there is an example of cuml.dask.neighbors, I want to try it with tiny data.
import cudf
import dask_cudf
from cuml.dask.neighbors import NearestNeighbors
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
cluster = LocalCUDACluster() #use 2 GPUs
client=Client(cluster)
df = cudf.DataFrame(
{
"a": range(20),
"b": reversed(range(20)),
"c": range(20)
}
)
print(df)
ddf = dask_cudf.from_cudf(df, npartitions=2) #use 2 GPUs
print(ddf.head(5))
nn = NearestNeighbors(n_neighbors=3, clinet=client)
print(ddf.values)
nn.fit(ddf.values)
distances, indices = nn.kneighbors(ddf.values)
print(distances.head(5))
print(indices.head(5))
Describe the problems or issues found in the documentation
I met a problem.
2024-04-17 16:57:09,399 - distributed.protocol.pickle - ERROR - Failed to serialize <ToPickle: HighLevelGraph with 1 layers.
<dask.highlevelgraph.HighLevelGraph object at 0x7f03e84c4cd0>
0. 139653757910208
>.
Traceback (most recent call last):
File "/home/wxl/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py", line 63, in dumps
result = pickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wxl/anaconda3/lib/python3.11/socket.py", line 274, in __getstate__
raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wxl/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py", line 68, in dumps
pickler.dump(x)
File "/home/wxl/anaconda3/lib/python3.11/socket.py", line 274, in __getstate__
raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wxl/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py", line 81, in dumps
...
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wxl/anaconda3/lib/python3.11/socket.py", line 274, in __getstate__
raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
Output is truncated. View as a [scrollable element](command:cellOutput.enableScrolling?abe105fa-a2b8-4161-b846-fbacefd7919d) or open in a [text editor](command:workbench.action.openLargeOutput?abe105fa-a2b8-4161-b846-fbacefd7919d). Adjust cell output [settings](command:workbench.action.openSettings?%5B%22%40tag%3AnotebookOutputLayout%22%5D)...
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:63](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:63), in dumps(x, buffer_callback, protocol)
[62](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:62) try:
---> [63](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:63) result = pickle.dumps(x, **dump_kwargs)
[64](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:64) except Exception:
File [~/anaconda3/lib/python3.11/socket.py:274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274), in socket.__getstate__(self)
[273](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:273) def __getstate__(self):
--> [274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274) raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:68](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:68), in dumps(x, buffer_callback, protocol)
[67](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:67) buffers.clear()
---> [68](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:68) pickler.dump(x)
[69](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:69) result = f.getvalue()
File [~/anaconda3/lib/python3.11/socket.py:274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274), in socket.__getstate__(self)
[273](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:273) def __getstate__(self):
--> [274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274) raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
...
--> [379](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:379) raise TypeError(msg, str_x) from exc
[380](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:380) else: # pragma: nocover
[381](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:381) raise ValueError(f"{on_error=}; expected 'message' or 'raise'")
Steps taken to verify documentation is incorrect
The example in README should work without any conditions I think. If not, I think it is incorrect.
Suggested fix for documentation
Give us an example with tiny data which can work with any two GPUs.
Steps taken to search for needed documentation
I ever tried dask-cuml years ago following the example in tests which was archived for a long time. Now I think cuml.dask should be much easier to use and the tests seem far more advanced to follow. But there are newbies like me still be a newbie never archived. So please keep a working example and let us play!
Hi @wuxianliang, thanks for the issue! Could you run this script https://github.com/rapidsai/cuml/blob/branch-24.06/print_env.sh and put the output of it here? This problem is unexpected and would help us triage
Click here to see environment details
**git*** Not inside a git repository ***OS Information*** DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS" PRETTY_NAME="Ubuntu 22.04.4 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.4 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy Linux stockai 5.19.0-50-generic #50-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 10 18:24:29 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux ***GPU Information*** Thu Apr 18 12:55:13 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.171.04 Driver Version: 535.171.04 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 Quadro M4000 Off | 00000000:02:00.0 On | N/A | | 49% 51C P0 44W / 120W | 615MiB / 8192MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ | 1 NVIDIA GeForce RTX 2070 Off | 00000000:83:00.0 On | N/A | | 30% 34C P8 10W / 175W | 86MiB / 8192MiB | 2% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 5658 G /usr/lib/xorg/Xorg 280MiB | | 0 N/A N/A 5968 G /usr/bin/gnome-shell 142MiB | | 0 N/A N/A 8373 G ...seed-version=20240416-180159.777000 75MiB | | 1 N/A N/A 5658 G /usr/lib/xorg/Xorg 85MiB | +---------------------------------------------------------------------------------------+ ***CPU*** 架构: x86_64 CPU 运行模式: 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual 字节序: Little Endian CPU: 32 在线 CPU 列表: 0-31 厂商 ID: GenuineIntel 型号名称: Intel(R) Xeon(R) CPU E5-2687W v2 @ 3.40GHz CPU 系列: 6 型号: 62 每个核的线程数: 2 每个座的核数: 8 座: 2 步进: 4 CPU 最大 MHz: 4000.0000 CPU 最小 MHz: 1200.0000 BogoMIPS: 6799.30 标记: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d 虚拟化: VT-x L1d 缓存: 512 KiB (16 instances) L1i 缓存: 512 KiB (16 instances) L2 缓存: 4 MiB (16 instances) L3 缓存: 50 MiB (2 instances) NUMA 节点: 2 NUMA 节点0 CPU: 0-7,16-23 NUMA 节点1 CPU: 8-15,24-31 Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable Vulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable Vulnerability Meltdown: Mitigation; PTI Vulnerability Mmio stale data: Unknown: No mitigations Vulnerability Retbleed: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected ***CMake*** ***g++*** /usr/bin/g++ g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ***nvcc*** /usr/local/cuda/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Aug_15_22:02:13_PDT_2023 Cuda compilation tools, release 12.2, V12.2.140 Build cuda_12.2.r12.2/compiler.33191640_0 ***Python*** /home/wxl/anaconda3/bin/python Python 3.11.7 ***Environment Variables*** PATH : /usr/local/cuda/bin:/home/wxl/anaconda3/bin:/home/wxl/anaconda3/condabin:/home/wxl/.nvm/versions/node/v21.7.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin LD_LIBRARY_PATH : /usr/local/cuda/lib64 NUMBAPRO_NVVM : NUMBAPRO_LIBDEVICE : CONDA_PREFIX : /home/wxl/anaconda3 PYTHON_PATH : ***conda packages*** /home/wxl/anaconda3/bin/conda # packages in environment at /home/wxl/anaconda3: # # Name Version Build Channel _anaconda_depends 2024.02 py311_mkl_1 _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu abseil-cpp 20211102.0 hd4dd3e8_0 aiobotocore 2.7.0 py311h06a4308_0 aiohttp 3.9.3 py311h5eee18b_0 aioitertools 0.7.1 pyhd3eb1b0_0 aiosignal 1.2.0 pyhd3eb1b0_0 alabaster 0.7.12 pyhd3eb1b0_0 altair 5.0.1 py311h06a4308_0 anaconda-anon-usage 0.4.3 py311hfc0e8ea_100 anaconda-catalogs 0.2.0 py311h06a4308_0 anaconda-client 1.12.3 py311h06a4308_0 anaconda-cloud-auth 0.1.4 py311h06a4308_0 anaconda-navigator 2.5.2 py311h06a4308_0 anaconda-project 0.11.1 py311h06a4308_0 anyio 4.2.0 py311h06a4308_0 aom 3.6.0 h6a678d5_0 appdirs 1.4.4 pyhd3eb1b0_0 archspec 0.2.3 pyhd3eb1b0_0 argon2-cffi 21.3.0 pyhd3eb1b0_0 argon2-cffi-bindings 21.2.0 py311h5eee18b_0 arrow 1.2.3 py311h06a4308_1 arrow-cpp 14.0.2 h374c478_1 astroid 2.14.2 py311h06a4308_0 astropy 5.3.4 py311hf4808d0_0 asttokens 2.0.5 pyhd3eb1b0_0 async-lru 2.0.4 py311h06a4308_0 atomicwrites 1.4.0 py_0 attrs 23.1.0 py311h06a4308_0 automat 20.2.0 py_0 autopep8 1.6.0 pyhd3eb1b0_1 aws-c-auth 0.6.19 h5eee18b_0 aws-c-cal 0.5.20 hdbd6064_0 aws-c-common 0.8.5 h5eee18b_0 aws-c-compression 0.2.16 h5eee18b_0 aws-c-event-stream 0.2.15 h6a678d5_0 aws-c-http 0.6.25 h5eee18b_0 aws-c-io 0.13.10 h5eee18b_0 aws-c-mqtt 0.7.13 h5eee18b_0 aws-c-s3 0.1.51 hdbd6064_0 aws-c-sdkutils 0.1.6 h5eee18b_0 aws-checksums 0.1.13 h5eee18b_0 aws-crt-cpp 0.18.16 h6a678d5_0 aws-sdk-cpp 1.10.55 h721c034_0 babel 2.11.0 py311h06a4308_0 backports 1.1 pyhd3eb1b0_0 backports.functools_lru_cache 1.6.4 pyhd3eb1b0_0 backports.tempfile 1.0 pyhd3eb1b0_1 backports.weakref 1.0.post1 py_1 bcrypt 3.2.0 py311h5eee18b_1 beautifulsoup4 4.12.2 py311h06a4308_0 binaryornot 0.4.4 pyhd3eb1b0_1 black 23.11.0 py311h06a4308_0 blas 1.0 mkl bleach 4.1.0 pyhd3eb1b0_0 blinker 1.6.2 py311h06a4308_0 blosc 1.21.3 h6a678d5_0 bokeh 3.3.4 py311h92b7b1e_0 boltons 23.0.0 py311h06a4308_0 boost-cpp 1.82.0 hdb19cb5_2 botocore 1.31.64 py311h06a4308_0 bottleneck 1.3.7 py311hf4808d0_0 brotli 1.0.9 h5eee18b_7 brotli-bin 1.0.9 h5eee18b_7 brotli-python 1.0.9 py311h6a678d5_7 brunsli 0.1 h2531618_0 bs4 0.0.2 pypi_0 pypi bzip2 1.0.8 h7b6447c_0 c-ares 1.19.1 h5eee18b_0 c-blosc2 2.12.0 h80c7b02_0 ca-certificates 2024.3.11 h06a4308_0 cachetools 4.2.2 pyhd3eb1b0_0 certifi 2024.2.2 py311h06a4308_0 cffi 1.16.0 py311h5eee18b_0 cfitsio 3.470 h5893167_7 chardet 4.0.0 py311h06a4308_1003 charls 2.2.0 h2531618_0 charset-normalizer 2.0.4 pyhd3eb1b0_0 click 8.1.7 py311h06a4308_0 click-plugins 1.1.1 pypi_0 pypi cligj 0.7.2 pypi_0 pypi cloudpickle 2.2.1 py311h06a4308_0 clyent 1.2.2 py311h06a4308_1 colorama 0.4.6 py311h06a4308_0 colorcet 3.0.1 py311h06a4308_0 comm 0.1.2 py311h06a4308_0 conda 24.3.0 py311h06a4308_0 conda-build 24.1.2 py311h06a4308_0 conda-content-trust 0.2.0 py311h06a4308_0 conda-index 0.4.0 pyhd3eb1b0_0 conda-libmamba-solver 24.1.0 pyhd3eb1b0_0 conda-pack 0.6.0 pyhd3eb1b0_0 conda-package-handling 2.2.0 py311h06a4308_0 conda-package-streaming 0.9.0 py311h06a4308_0 conda-repo-cli 1.0.75 py311h06a4308_0 conda-token 0.4.0 pyhd3eb1b0_0 conda-verify 3.4.2 py_1 constantly 23.10.4 py311h06a4308_0 contourpy 1.2.0 py311hdb19cb5_0 cookiecutter 2.5.0 py311h06a4308_0 cryptography 42.0.2 py311hdda0065_0 cssselect 1.2.0 py311h06a4308_0 cuda-python 12.4.0 pypi_0 pypi cudf-cu12 24.4.0 pypi_0 pypi cugraph-cu12 24.4.0 pypi_0 pypi cuml-cu12 24.4.0 pypi_0 pypi cupy-cuda12x 13.0.0 pypi_0 pypi curl 8.5.0 hdbd6064_0 cuspatial-cu12 24.4.0 pypi_0 pypi cuvs-cu12 24.4.0 pypi_0 pypi cuxfilter-cu12 24.4.1 pypi_0 pypi cycler 0.11.0 pyhd3eb1b0_0 cyrus-sasl 2.1.28 h52b45da_1 cytoolz 0.12.2 py311h5eee18b_0 dask 2024.1.1 pypi_0 pypi dask-cuda 24.4.0 pypi_0 pypi dask-cudf-cu12 24.4.0 pypi_0 pypi dask-expr 0.4.0 pypi_0 pypi datashader 0.16.0 py311h06a4308_0 dav1d 1.2.1 h5eee18b_0 dbus 1.13.18 hb2f20db_0 debugpy 1.6.7 py311h6a678d5_0 decorator 5.1.1 pyhd3eb1b0_0 defusedxml 0.7.1 pyhd3eb1b0_0 diff-match-patch 20200713 pyhd3eb1b0_0 dill 0.3.7 py311h06a4308_0 distributed 2024.1.1 pypi_0 pypi distro 1.8.0 py311h06a4308_0 docstring-to-markdown 0.11 py311h06a4308_0 docutils 0.18.1 py311h06a4308_3 entrypoints 0.4 py311h06a4308_0 et_xmlfile 1.1.0 py311h06a4308_0 executing 0.8.3 pyhd3eb1b0_0 expat 2.5.0 h6a678d5_0 fastrlock 0.8.2 pypi_0 pypi filelock 3.13.1 py311h06a4308_0 fiona 1.9.6 pypi_0 pypi flake8 6.0.0 py311h06a4308_0 flask 2.2.5 py311h06a4308_0 fmt 9.1.0 hdb19cb5_0 fontconfig 2.14.1 h4c34cd2_2 fonttools 4.25.0 pyhd3eb1b0_0 fqdn 1.5.1 pypi_0 pypi freetype 2.12.1 h4a9f257_0 frozenlist 1.4.0 py311h5eee18b_0 fsspec 2023.10.0 py311h06a4308_0 future 0.18.3 py311h06a4308_0 gensim 4.3.0 py311hba01205_1 geopandas 0.14.3 pypi_0 pypi gflags 2.2.2 h6a678d5_1 giflib 5.2.1 h5eee18b_3 gitdb 4.0.7 pyhd3eb1b0_0 gitpython 3.1.37 py311h06a4308_0 glib 2.78.4 h6a678d5_0 glib-tools 2.78.4 h6a678d5_0 glog 0.5.0 h6a678d5_1 gmp 6.2.1 h295c915_3 gmpy2 2.1.2 py311hc9b5ff0_0 gpudtw 0.13 pypi_0 pypi greenlet 3.0.1 py311h6a678d5_0 grpc-cpp 1.48.2 he1ff14a_1 gst-plugins-base 1.14.1 h6a678d5_1 gstreamer 1.14.1 h5eee18b_1 h5py 3.9.0 py311hdd6beaf_0 hdf5 1.12.1 h2b7332f_3 heapdict 1.0.1 pyhd3eb1b0_0 holoviews 1.18.3 py311h06a4308_0 hvplot 0.9.2 py311h06a4308_0 hyperlink 21.0.0 pyhd3eb1b0_0 icu 73.1 h6a678d5_0 idna 3.4 py311h06a4308_0 imagecodecs 2023.1.23 py311h8105a5c_0 imageio 2.33.1 py311h06a4308_0 imagesize 1.4.1 py311h06a4308_0 imbalanced-learn 0.11.0 py311h06a4308_1 importlib-metadata 7.0.1 py311h06a4308_0 importlib_metadata 7.0.1 hd3eb1b0_0 incremental 22.10.0 pyhd3eb1b0_0 inflection 0.5.1 py311h06a4308_0 iniconfig 1.1.1 pyhd3eb1b0_0 intake 0.6.8 py311h06a4308_0 intel-openmp 2023.1.0 hdb19cb5_46306 intervaltree 3.1.0 pyhd3eb1b0_0 ipykernel 6.28.0 py311h06a4308_0 ipython 8.20.0 py311h06a4308_0 ipython_genutils 0.2.0 pyhd3eb1b0_1 ipywidgets 7.6.5 pyhd3eb1b0_2 isoduration 20.11.0 pypi_0 pypi isort 5.9.3 pyhd3eb1b0_0 itemadapter 0.3.0 pyhd3eb1b0_0 itemloaders 1.1.0 py311h06a4308_0 itsdangerous 2.0.1 pyhd3eb1b0_0 jaraco.classes 3.2.1 pyhd3eb1b0_0 jedi 0.18.1 py311h06a4308_1 jeepney 0.7.1 pyhd3eb1b0_0 jellyfish 1.0.1 py311hb02cf49_0 jinja2 3.1.3 py311h06a4308_0 jmespath 1.0.1 py311h06a4308_0 joblib 1.2.0 py311h06a4308_0 jpeg 9e h5eee18b_1 jq 1.6 h27cfd23_1000 json5 0.9.6 pyhd3eb1b0_0 jsonpatch 1.32 pyhd3eb1b0_0 jsonpointer 2.1 pyhd3eb1b0_0 jsonschema 4.19.2 py311h06a4308_0 jsonschema-specifications 2023.7.1 py311h06a4308_0 jupyter 1.0.0 py311h06a4308_9 jupyter-lsp 2.2.0 py311h06a4308_0 jupyter-server-proxy 4.1.2 pypi_0 pypi jupyter_client 8.6.0 py311h06a4308_0 jupyter_console 6.6.3 py311h06a4308_0 jupyter_core 5.5.0 py311h06a4308_0 jupyter_events 0.8.0 py311h06a4308_0 jupyter_server 2.10.0 py311h06a4308_0 jupyter_server_terminals 0.4.4 py311h06a4308_1 jupyterlab 4.0.11 py311h06a4308_0 jupyterlab-variableinspector 3.1.0 py311h06a4308_0 jupyterlab_pygments 0.1.2 py_0 jupyterlab_server 2.25.1 py311h06a4308_0 jupyterlab_widgets 3.0.9 py311h06a4308_0 jxrlib 1.1 h7b6447c_2 keyring 23.13.1 py311h06a4308_0 kiwisolver 1.4.4 py311h6a678d5_0 krb5 1.20.1 h143b758_1 lazy-object-proxy 1.6.0 py311h5eee18b_0 lazy_loader 0.3 py311h06a4308_0 lcms2 2.12 h3be6417_0 ld_impl_linux-64 2.38 h1181459_1 lerc 3.0 h295c915_0 libaec 1.0.4 he6710b0_1 libarchive 3.6.2 h6ac8c49_2 libavif 0.11.1 h5eee18b_0 libboost 1.82.0 h109eef0_2 libbrotlicommon 1.0.9 h5eee18b_7 libbrotlidec 1.0.9 h5eee18b_7 libbrotlienc 1.0.9 h5eee18b_7 libclang 14.0.6 default_hc6dbbc7_1 libclang13 14.0.6 default_he11475f_1 libcups 2.4.2 h2d74bed_1 libcurl 8.5.0 h251f7ec_0 libdeflate 1.17 h5eee18b_1 libedit 3.1.20230828 h5eee18b_0 libev 4.33 h7f8727e_1 libevent 2.1.12 hdbd6064_1 libffi 3.4.4 h6a678d5_0 libgcc-ng 11.2.0 h1234567_1 libgfortran-ng 11.2.0 h00389a5_1 libgfortran5 11.2.0 h1234567_1 libglib 2.78.4 hdc74915_0 libgomp 11.2.0 h1234567_1 libiconv 1.16 h7f8727e_2 liblief 0.12.3 h6a678d5_0 libllvm14 14.0.6 hdb19cb5_3 libmamba 1.5.6 haf1ee3a_0 libmambapy 1.5.6 py311h2dafd23_0 libnghttp2 1.57.0 h2d74bed_0 libpng 1.6.39 h5eee18b_0 libpq 12.17 hdbd6064_0 libprotobuf 3.20.3 he621ea3_0 libsodium 1.0.18 h7b6447c_0 libsolv 0.7.24 he621ea3_0 libspatialindex 1.9.3 h2531618_0 libssh2 1.10.0 hdbd6064_2 libstdcxx-ng 11.2.0 h1234567_1 libthrift 0.15.0 h1795dd8_2 libtiff 4.5.1 h6a678d5_0 libuuid 1.41.5 h5eee18b_0 libwebp-base 1.3.2 h5eee18b_0 libxcb 1.15 h7f8727e_0 libxkbcommon 1.0.1 h5eee18b_1 libxml2 2.10.4 hf1b16e4_1 libxslt 1.1.37 h5eee18b_1 libzopfli 1.0.3 he6710b0_0 linkify-it-py 2.0.0 py311h06a4308_0 llvmlite 0.42.0 py311h6a678d5_0 locket 1.0.0 py311h06a4308_0 lxml 4.9.3 py311hdbbb534_0 lz4 4.3.2 py311h5eee18b_0 lz4-c 1.9.4 h6a678d5_0 lzo 2.10 h7b6447c_2 mako 1.3.3 pypi_0 pypi markdown 3.4.1 py311h06a4308_0 markdown-it-py 2.2.0 py311h06a4308_1 markupsafe 2.1.3 py311h5eee18b_0 matplotlib 3.8.0 py311h06a4308_0 matplotlib-base 3.8.0 py311ha02d727_0 matplotlib-inline 0.1.6 py311h06a4308_0 mccabe 0.7.0 pyhd3eb1b0_0 mdit-py-plugins 0.3.0 py311h06a4308_0 mdurl 0.1.0 py311h06a4308_0 menuinst 2.0.2 py311h06a4308_0 mistune 2.0.4 py311h06a4308_0 mkl 2023.1.0 h213fc3f_46344 mkl-service 2.4.0 py311h5eee18b_1 mkl_fft 1.3.8 py311h5eee18b_0 mkl_random 1.2.4 py311hdb19cb5_0 modin 0.29.0 pypi_0 pypi more-itertools 10.1.0 py311h06a4308_0 mpc 1.1.0 h10f8cd9_1 mpfr 4.0.2 hb69a4c5_1 mpmath 1.3.0 py311h06a4308_0 msgpack 1.0.8 pypi_0 pypi multidict 6.0.4 py311h5eee18b_0 multipledispatch 0.6.0 py311h06a4308_0 munkres 1.1.4 py_0 mypy 1.8.0 py311h5eee18b_0 mypy_extensions 1.0.0 py311h06a4308_0 mysql 5.7.24 h721c034_2 navigator-updater 0.4.0 py311h06a4308_1 nbclient 0.8.0 py311h06a4308_0 nbconvert 7.10.0 py311h06a4308_0 nbformat 5.9.2 py311h06a4308_0 ncurses 6.4 h6a678d5_0 nest-asyncio 1.6.0 py311h06a4308_0 networkx 3.1 py311h06a4308_0 nltk 3.8.1 py311h06a4308_0 notebook 7.0.8 py311h06a4308_0 notebook-shim 0.2.3 py311h06a4308_0 nspr 4.35 h6a678d5_0 nss 3.89.1 h6a678d5_0 numba 0.59.0 py311ha02d727_0 numexpr 2.8.7 py311h65dcdc2_0 numpy 1.26.4 py311h08b1b3b_0 numpy-base 1.26.4 py311hf175353_0 numpydoc 1.5.0 py311h06a4308_0 nvtx 0.2.10 pypi_0 pypi oniguruma 6.9.7.1 h27cfd23_0 openjpeg 2.4.0 h3ad879b_0 openpyxl 3.0.10 py311h5eee18b_0 openssl 3.0.13 h7f8727e_0 orc 1.7.4 hb3bc3d3_1 overrides 7.4.0 py311h06a4308_0 packaging 23.1 py311h06a4308_0 pandas 2.2.1 pypi_0 pypi pandocfilters 1.5.0 pyhd3eb1b0_0 panel 1.3.8 py311h06a4308_0 param 2.0.2 py311h06a4308_0 parsel 1.8.1 py311h06a4308_0 parso 0.8.3 pyhd3eb1b0_0 partd 1.4.1 py311h06a4308_0 patch 2.7.6 h7b6447c_1001 patchelf 0.17.2 h6a678d5_0 pathlib 1.0.1 pyhd3eb1b0_1 pathspec 0.10.3 py311h06a4308_0 patsy 0.5.3 py311h06a4308_0 pcre2 10.42 hebb0a14_0 pexpect 4.8.0 pyhd3eb1b0_3 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 10.2.0 py311h5eee18b_0 pip 23.3.1 py311h06a4308_0 pkce 1.0.3 py311h06a4308_0 pkginfo 1.9.6 py311h06a4308_0 platformdirs 3.10.0 py311h06a4308_0 plotly 5.9.0 py311h06a4308_0 pluggy 1.0.0 py311h06a4308_1 ply 3.11 py311h06a4308_0 prometheus_client 0.14.1 py311h06a4308_0 prompt-toolkit 3.0.43 py311h06a4308_0 prompt_toolkit 3.0.43 hd3eb1b0_0 protego 0.1.16 py_0 protobuf 4.25.3 pypi_0 pypi psutil 5.9.0 py311h5eee18b_0 psycopg2 2.9.9 pypi_0 pypi ptyprocess 0.7.0 pyhd3eb1b0_2 pure_eval 0.2.2 pyhd3eb1b0_0 py-cpuinfo 9.0.0 py311h06a4308_0 py-lief 0.12.3 py311h6a678d5_0 pyarrow 14.0.2 py311hb6e97c4_0 pyasn1 0.4.8 pyhd3eb1b0_0 pyasn1-modules 0.2.8 py_0 pybind11-abi 4 hd3eb1b0_1 pycodestyle 2.10.0 py311h06a4308_0 pycosat 0.6.6 py311h5eee18b_0 pycparser 2.21 pyhd3eb1b0_0 pyct 0.5.0 py311h06a4308_0 pycuda 2024.1 pypi_0 pypi pycurl 7.45.2 py311hdbd6064_1 pydantic 1.10.12 py311h5eee18b_1 pydeck 0.8.0 py311h06a4308_2 pydispatcher 2.0.5 py311h06a4308_2 pydocstyle 6.3.0 py311h06a4308_0 pyerfa 2.0.0 py311h5eee18b_0 pyflakes 3.0.1 py311h06a4308_0 pygments 2.15.1 py311h06a4308_1 pyjwt 2.4.0 py311h06a4308_0 pylibcugraph-cu12 24.4.0 pypi_0 pypi pylibraft-cu12 24.4.0 pypi_0 pypi pylint 2.16.2 py311h06a4308_0 pylint-venv 2.3.0 py311h06a4308_0 pyls-spyder 0.4.0 pyhd3eb1b0_0 pynvjitlink-cu12 0.1.14 pypi_0 pypi pynvml 11.4.1 pypi_0 pypi pyodbc 5.0.1 py311h6a678d5_0 pyopencl 2024.1 pypi_0 pypi pyopenssl 24.0.0 py311h06a4308_0 pyparsing 3.0.9 py311h06a4308_0 pyproj 3.6.1 pypi_0 pypi pyqt 5.15.10 py311h6a678d5_0 pyqt5-sip 12.13.0 py311h5eee18b_0 pyqtwebengine 5.15.10 py311h6a678d5_0 pysocks 1.7.1 py311h06a4308_0 pytables 3.9.2 py311h9d13977_0 pytest 7.4.0 py311h06a4308_0 python 3.11.7 h955ad1f_0 python-dateutil 2.8.2 pyhd3eb1b0_0 python-dotenv 0.21.0 py311h06a4308_0 python-fastjsonschema 2.16.2 py311h06a4308_0 python-json-logger 2.0.7 py311h06a4308_0 python-libarchive-c 2.9 pyhd3eb1b0_1 python-lmdb 1.4.1 py311h6a678d5_0 python-lsp-black 1.2.1 py311h06a4308_0 python-lsp-jsonrpc 1.0.0 pyhd3eb1b0_0 python-lsp-server 1.7.2 py311h06a4308_0 python-slugify 5.0.2 pyhd3eb1b0_0 python-snappy 0.6.1 py311h6a678d5_0 python-tzdata 2023.3 pyhd3eb1b0_0 pytoolconfig 1.2.6 py311h06a4308_0 pytools 2024.1.1 pypi_0 pypi pytz 2023.3.post1 py311h06a4308_0 pyviz_comms 3.0.0 py311h06a4308_0 pywavelets 1.5.0 py311hf4808d0_0 pyxdg 0.27 pyhd3eb1b0_0 pyyaml 6.0.1 py311h5eee18b_0 pyzmq 25.1.2 py311h6a678d5_0 qdarkstyle 3.0.2 pyhd3eb1b0_0 qstylizer 0.2.2 py311h06a4308_0 qt-main 5.15.2 h53bd1ea_10 qt-webengine 5.15.9 h9ab4d14_7 qtawesome 1.2.2 py311h06a4308_0 qtconsole 5.4.2 py311h06a4308_0 qtpy 2.4.1 py311h06a4308_0 queuelib 1.6.2 py311h06a4308_0 raft-dask-cu12 24.4.0 pypi_0 pypi rapids-dask-dependency 24.4.1 pypi_0 pypi re2 2022.04.01 h295c915_0 readline 8.2 h5eee18b_0 referencing 0.30.2 py311h06a4308_0 regex 2023.10.3 py311h5eee18b_0 reproc 14.2.4 h295c915_1 reproc-cpp 14.2.4 h295c915_1 requests 2.31.0 py311h06a4308_1 requests-file 1.5.1 pyhd3eb1b0_0 requests-toolbelt 1.0.0 py311h06a4308_0 rfc3339-validator 0.1.4 py311h06a4308_0 rfc3986-validator 0.1.1 py311h06a4308_0 rich 13.3.5 py311h06a4308_0 rmm-cu12 24.4.0 pypi_0 pypi rope 1.7.0 py311h06a4308_0 rpds-py 0.10.6 py311hb02cf49_0 rtree 1.0.1 py311h06a4308_0 ruamel.yaml 0.17.21 py311h5eee18b_0 ruamel_yaml 0.17.21 py311h5eee18b_0 s2n 1.3.27 hdbd6064_0 s3fs 2023.10.0 py311h06a4308_0 scikit-image 0.22.0 py311ha02d727_0 scikit-learn 1.2.2 py311h6a678d5_1 scipy 1.11.4 py311h08b1b3b_0 scrapy 2.8.0 py311h06a4308_0 seaborn 0.12.2 py311h06a4308_0 secretstorage 3.3.1 py311h06a4308_1 semver 2.13.0 pyhd3eb1b0_0 send2trash 1.8.2 py311h06a4308_0 service_identity 18.1.0 pyhd3eb1b0_1 setuptools 68.2.2 py311h06a4308_0 shapely 2.0.3 pypi_0 pypi simpervisor 1.0.0 pypi_0 pypi simplejson 3.19.2 pypi_0 pypi sip 6.7.12 py311h6a678d5_0 six 1.16.0 pyhd3eb1b0_1 smart_open 5.2.1 py311h06a4308_0 smmap 4.0.0 pyhd3eb1b0_0 snappy 1.1.10 h6a678d5_1 sniffio 1.3.0 py311h06a4308_0 snowballstemmer 2.2.0 pyhd3eb1b0_0 sortedcontainers 2.4.0 pyhd3eb1b0_0 soupsieve 2.5 py311h06a4308_0 sphinx 5.0.2 py311h06a4308_0 sphinxcontrib-applehelp 1.0.2 pyhd3eb1b0_0 sphinxcontrib-devhelp 1.0.2 pyhd3eb1b0_0 sphinxcontrib-htmlhelp 2.0.0 pyhd3eb1b0_0 sphinxcontrib-jsmath 1.0.1 pyhd3eb1b0_0 sphinxcontrib-qthelp 1.0.3 pyhd3eb1b0_0 sphinxcontrib-serializinghtml 1.1.5 pyhd3eb1b0_0 spyder 5.4.3 py311h06a4308_1 spyder-kernels 2.4.4 py311h06a4308_0 sqlalchemy 2.0.25 py311h5eee18b_0 sqlite 3.41.2 h5eee18b_0 stack_data 0.2.0 pyhd3eb1b0_0 statsmodels 0.14.0 py311hf4808d0_0 streamlit 1.30.0 py311h06a4308_0 sympy 1.12 py311h06a4308_0 tabulate 0.9.0 py311h06a4308_0 tbb 2021.8.0 hdb19cb5_0 tblib 1.7.0 pyhd3eb1b0_0 tenacity 8.2.2 py311h06a4308_0 terminado 0.17.1 py311h06a4308_0 text-unidecode 1.3 pyhd3eb1b0_0 textdistance 4.2.1 pyhd3eb1b0_0 threadpoolctl 2.2.0 pyh0d69192_0 three-merge 0.1.1 pyhd3eb1b0_0 tifffile 2023.4.12 py311h06a4308_0 tinycss2 1.2.1 py311h06a4308_0 tk 8.6.12 h1ccaba5_0 tldextract 3.2.0 pyhd3eb1b0_0 toml 0.10.2 pyhd3eb1b0_0 tomlkit 0.11.1 py311h06a4308_0 toolz 0.12.0 py311h06a4308_0 tornado 6.3.3 py311h5eee18b_0 tqdm 4.65.0 py311h92b7b1e_0 traitlets 5.7.1 py311h06a4308_0 treelite 4.1.2 pypi_0 pypi truststore 0.8.0 py311h06a4308_0 tslearn 0.6.3 pypi_0 pypi tushare 1.4.6 pypi_0 pypi twisted 23.10.0 py311h06a4308_0 typing-extensions 4.9.0 py311h06a4308_1 typing_extensions 4.9.0 py311h06a4308_1 tzdata 2023d h04d1e81_0 tzlocal 2.1 py311h06a4308_1 uc-micro-py 1.0.1 py311h06a4308_0 ucx-py-cu12 0.37.0 pypi_0 pypi ujson 5.4.0 py311h6a678d5_0 unidecode 1.2.0 pyhd3eb1b0_0 unixodbc 2.3.11 h5eee18b_0 uri-template 1.3.0 pypi_0 pypi urllib3 2.0.7 py311h06a4308_0 utf8proc 2.6.1 h5eee18b_1 validators 0.18.2 pyhd3eb1b0_0 w3lib 2.1.2 py311h06a4308_0 watchdog 2.1.6 py311h06a4308_0 wcwidth 0.2.5 pyhd3eb1b0_0 webcolors 1.13 pypi_0 pypi webencodings 0.5.1 py311h06a4308_1 websocket-client 0.57.0 pypi_0 pypi werkzeug 2.2.3 py311h06a4308_0 whatthepatch 1.0.2 py311h06a4308_0 wheel 0.41.2 py311h06a4308_0 widgetsnbextension 3.5.2 py311h06a4308_1 wrapt 1.14.1 py311h5eee18b_0 wurlitzer 3.0.2 py311h06a4308_0 xarray 2023.6.0 py311h06a4308_0 xyzservices 2022.9.0 py311h06a4308_1 xz 5.4.5 h5eee18b_0 yaml 0.2.5 h7b6447c_0 yaml-cpp 0.8.0 h6a678d5_0 yapf 0.31.0 pyhd3eb1b0_0 yarl 1.9.3 py311h5eee18b_0 zeromq 4.3.5 h6a678d5_0 zfp 1.0.0 h6a678d5_0 zict 3.0.0 py311h06a4308_0 zipp 3.17.0 py311h06a4308_0 zlib 1.2.13 h5eee18b_0 zlib-ng 2.0.7 h5eee18b_0 zope 1.0 py311h06a4308_1 zope.interface 5.4.0 py311h5eee18b_0 zstandard 0.19.0 py311h5eee18b_0 zstd 1.5.5 hc292b87_0
Sorry for my carelessness. I am testing the code with a Maxwell card and a 2070. It is my fault. I will change to use 4 Tesla V100. If I succeed, I will close this issue.
Thanks for the update @wuxianliang! The Maxwell card unfortunately is unsupported, which could be the root of the issue indeed, but glad to look deeper if you still run into problems!
Now I am trying to use one 2070 and one 3080ti and run the program as .ipynb in VS code, the problem appears still. Because the water-cooler system has not been finished, I will try 4x V100 later. My working condition now is temporary just for trying the distributed codes, I think rarely people use it like me. I will report 4x V100 case weeks later.
import cudf
import dask_cudf
from cuml.dask.neighbors import NearestNeighbors
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
cluster = LocalCUDACluster() #use 2 GPUs
client=Client(cluster)
print(client)
df = cudf.DataFrame(
{
"a": range(20),
"b": reversed(range(20)),
"c": range(20)
}
)
print(df)
ddf = dask_cudf.from_cudf(df, npartitions=2) #use 2 GPUs
print(ddf.head(5))
nn = NearestNeighbors(n_neighbors=3, clinet=client)
print(ddf.values)
nn.fit(ddf) #.values
distances, indices = nn.kneighbors(ddf) #.values
print(distances.head(5))
print(indices.head(5))
<Client: 'tcp://127.0.0.1:40715' processes=2 threads=2, memory=251.85 GiB>
a b c
0 0 19 0
1 1 18 1
2 2 17 2
3 3 16 3
4 4 15 4
5 5 14 5
6 6 13 6
7 7 12 7
8 8 11 8
9 9 10 9
10 10 9 10
11 11 8 11
12 12 7 12
13 13 6 13
14 14 5 14
15 15 4 15
16 16 3 16
17 17 2 17
18 18 1 18
19 19 0 19
a b c
0 0 19 0
1 1 18 1
2 2 17 2
3 3 16 3
4 4 15 4
dask.array<values, shape=(nan, 3), dtype=int64, chunksize=(nan, 3), chunktype=numpy.ndarray>
2024-04-24 09:02:22,616 - distributed.protocol.pickle - ERROR - Failed to serialize <ToPickle: HighLevelGraph with 1 layers.
<dask.highlevelgraph.HighLevelGraph object at 0x7f2ec9733890>
0. 139838925101888
>.
Traceback (most recent call last):
File "/home/wxl/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py", line 63, in dumps
result = pickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wxl/anaconda3/lib/python3.11/socket.py", line 274, in __getstate__
raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wxl/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py", line 68, in dumps
pickler.dump(x)
File "/home/wxl/anaconda3/lib/python3.11/socket.py", line 274, in __getstate__
raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wxl/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py", line 81, in dumps
result = cloudpickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wxl/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/wxl/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py", line 632, in dump
return Pickler.dump(self, obj)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wxl/anaconda3/lib/python3.11/socket.py", line 274, in __getstate__
raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:63](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:63), in dumps(x, buffer_callback, protocol)
[62](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:62) try:
---> [63](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:63) result = pickle.dumps(x, **dump_kwargs)
[64](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:64) except Exception:
File [~/anaconda3/lib/python3.11/socket.py:274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274), in socket.__getstate__(self)
[273](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:273) def __getstate__(self):
--> [274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274) raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:68](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:68), in dumps(x, buffer_callback, protocol)
[67](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:67) buffers.clear()
---> [68](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:68) pickler.dump(x)
[69](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:69) result = f.getvalue()
File [~/anaconda3/lib/python3.11/socket.py:274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274), in socket.__getstate__(self)
[273](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:273) def __getstate__(self):
--> [274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274) raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:353](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:353), in serialize(x, serializers, on_error, context, iterate_collection)
[352](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:352) try:
--> [353](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:353) header, frames = dumps(x, context=context) if wants_context else dumps(x)
[354](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:354) header["serializer"] = name
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:76](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:76), in pickle_dumps(x, context)
[74](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:74) writeable.append(not f.readonly)
---> [76](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:76) frames[0] = pickle.dumps(
[77](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:77) x,
[78](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:78) buffer_callback=buffer_callback,
[79](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:79) protocol=context.get("pickle-protocol", None) if context else None,
[80](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:80) )
[81](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:81) header = {
[82](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:82) "serializer": "pickle",
[83](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:83) "writeable": tuple(writeable),
[84](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:84) }
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:81](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:81), in dumps(x, buffer_callback, protocol)
[80](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:80) buffers.clear()
---> [81](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:81) result = cloudpickle.dumps(x, **dump_kwargs)
[82](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/pickle.py:82) except Exception:
File [~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:73](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:73), in dumps(obj, protocol, buffer_callback)
[70](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:70) cp = CloudPickler(
[71](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:71) file, protocol=protocol, buffer_callback=buffer_callback
[72](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:72) )
---> [73](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:73) cp.dump(obj)
[74](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:74) return file.getvalue()
File [~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:632](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:632), in CloudPickler.dump(self, obj)
[631](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:631) try:
--> [632](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:632) return Pickler.dump(self, obj)
[633](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cloudpickle/cloudpickle_fast.py:633) except RuntimeError as e:
File [~/anaconda3/lib/python3.11/socket.py:274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274), in socket.__getstate__(self)
[273](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:273) def __getstate__(self):
--> [274](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/socket.py:274) raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
TypeError: cannot pickle 'socket' object
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
Cell In[1], [line 24](vscode-notebook-cell:?execution_count=1&line=24)
[22](vscode-notebook-cell:?execution_count=1&line=22) print(ddf.values)
[23](vscode-notebook-cell:?execution_count=1&line=23) nn.fit(ddf)
---> [24](vscode-notebook-cell:?execution_count=1&line=24) distances, indices = nn.kneighbors(ddf)
[25](vscode-notebook-cell:?execution_count=1&line=25) print(distances.head(5))
[26](vscode-notebook-cell:?execution_count=1&line=26) print(indices.head(5))
File [~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:324](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:324), in NearestNeighbors.kneighbors(self, X, n_neighbors, return_distance, _return_futures)
[317](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:317) comms = NearestNeighbors._build_comms(
[318](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:318) self.X_handler, query_handler, self.streams_per_handle
[319](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:319) )
[321](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:321) """
[322](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:322) Initialize models on workers
[323](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:323) """
--> [324](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:324) nn_models = self._create_models(comms)
[326](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:326) """
[327](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:327) Perform model query
[328](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:328) """
[329](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:329) nn_fit, out_d_futures, out_i_futures = self._query_models(
[330](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:330) n_neighbors, comms, nn_models, self.X_handler, query_handler
[331](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:331) )
File [~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:176](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:176), in NearestNeighbors._create_models(self, comms)
[171](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:171) """
[172](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:172) Each Dask worker creates a single model
[173](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:173) """
[174](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:174) key = uuid1()
[175](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:175) nn_models = dict(
--> [176](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:176) [
[177](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:177) (
[178](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:178) worker,
[179](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:179) self.client.submit(
[180](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:180) NearestNeighbors._func_create_model,
[181](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:181) comms.sessionId,
[182](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:182) **self.kwargs,
[183](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:183) workers=[worker],
[184](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:184) key="%s-%s" % (key, idx),
[185](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:185) ),
[186](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:186) )
[187](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:187) for idx, worker in enumerate(comms.worker_addresses)
[188](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:188) ]
[189](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:189) )
[191](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:191) return nn_models
File [~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:179](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:179), in <listcomp>(.0)
[171](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:171) """
[172](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:172) Each Dask worker creates a single model
[173](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:173) """
[174](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:174) key = uuid1()
[175](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:175) nn_models = dict(
[176](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:176) [
[177](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:177) (
[178](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:178) worker,
--> [179](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:179) self.client.submit(
[180](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:180) NearestNeighbors._func_create_model,
[181](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:181) comms.sessionId,
[182](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:182) **self.kwargs,
[183](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:183) workers=[worker],
[184](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:184) key="%s-%s" % (key, idx),
[185](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:185) ),
[186](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:186) )
[187](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:187) for idx, worker in enumerate(comms.worker_addresses)
[188](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:188) ]
[189](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:189) )
[191](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/cuml/dask/neighbors/nearest_neighbors.py:191) return nn_models
File [~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1961](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1961), in Client.submit(self, func, key, workers, resources, retries, priority, fifo_timeout, allow_other_workers, actor, actors, pure, *args, **kwargs)
[1958](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1958) else:
[1959](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1959) dsk = {key: (func,) + tuple(args)}
-> [1961](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1961) futures = self._graph_to_futures(
[1962](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1962) dsk,
[1963](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1963) [key],
[1964](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1964) workers=workers,
[1965](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1965) allow_other_workers=allow_other_workers,
[1966](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1966) internal_priority={key: 0},
[1967](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1967) user_priority=priority,
[1968](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1968) resources=resources,
[1969](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1969) retries=retries,
[1970](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1970) fifo_timeout=fifo_timeout,
[1971](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1971) actors=actor,
[1972](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1972) )
[1974](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1974) logger.debug("Submit %s(...), %s", funcname(func), key)
[1976](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:1976) return futures[key]
File [~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3158](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3158), in Client._graph_to_futures(self, dsk, keys, workers, allow_other_workers, internal_priority, user_priority, resources, retries, fifo_timeout, actors)
[3155](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3155) from distributed.protocol import serialize
[3156](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3156) from distributed.protocol.serialize import ToPickle
-> [3158](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3158) header, frames = serialize(ToPickle(dsk), on_error="raise")
[3160](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3160) pickled_size = sum(map(nbytes, [header] + frames))
[3161](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/client.py:3161) if pickled_size > 10_000_000:
File [~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:379](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:379), in serialize(x, serializers, on_error, context, iterate_collection)
[377](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:377) except Exception:
[378](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:378) raise TypeError(msg) from exc
--> [379](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:379) raise TypeError(msg, str_x) from exc
[380](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:380) else: # pragma: nocover
[381](https://file+.vscode-resource.vscode-cdn.net/home/wxl/trading-platform/notes/explore/~/anaconda3/lib/python3.11/site-packages/distributed/protocol/serialize.py:381) raise ValueError(f"{on_error=}; expected 'message' or 'raise'")
TypeError: ('Could not serialize object of type HighLevelGraph', '<ToPickle: HighLevelGraph with 1 layers.\n<dask.highlevelgraph.HighLevelGraph object at 0x7f2ec9733890>\n 0. 139838925101888\n>')
Click here to see environment details
**git*** Not inside a git repository ***OS Information*** DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS" PRETTY_NAME="Ubuntu 22.04.4 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.4 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy Linux stockai 5.19.0-50-generic #50-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 10 18:24:29 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux ***GPU Information*** Wed Apr 24 09:27:56 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.171.04 Driver Version: 535.171.04 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA Graphics Device Off | 00000000:02:00.0 Off | N/A | | 30% 41C P2 93W / 350W | 895MiB / 20480MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ | 1 NVIDIA GeForce RTX 2070 Off | 00000000:83:00.0 On | N/A | | 35% 51C P0 46W / 175W | 893MiB / 8192MiB | 1% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 4156 G /usr/lib/xorg/Xorg 4MiB | | 0 N/A N/A 7889 C /home/wxl/anaconda3/bin/python 258MiB | | 0 N/A N/A 8620 C /home/wxl/anaconda3/bin/python 308MiB | | 0 N/A N/A 9504 C /home/wxl/anaconda3/bin/python 308MiB | | 1 N/A N/A 4156 G /usr/lib/xorg/Xorg 342MiB | | 1 N/A N/A 4460 G /usr/bin/gnome-shell 69MiB | | 1 N/A N/A 6649 G ...erProcess --variations-seed-version 50MiB | | 1 N/A N/A 8033 G ...seed-version=20240423-050130.785000 88MiB | | 1 N/A N/A 8621 C /home/wxl/anaconda3/bin/python 148MiB | | 1 N/A N/A 9505 C /home/wxl/anaconda3/bin/python 148MiB | +---------------------------------------------------------------------------------------+ ***CPU*** 架构: x86_64 CPU 运行模式: 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual 字节序: Little Endian CPU: 32 在线 CPU 列表: 0-31 厂商 ID: GenuineIntel 型号名称: Intel(R) Xeon(R) CPU E5-2687W v2 @ 3.40GHz CPU 系列: 6 型号: 62 每个核的线程数: 2 每个座的核数: 8 座: 2 步进: 4 CPU 最大 MHz: 4000.0000 CPU 最小 MHz: 1200.0000 BogoMIPS: 6800.15 标记: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d 虚拟化: VT-x L1d 缓存: 512 KiB (16 instances) L1i 缓存: 512 KiB (16 instances) L2 缓存: 4 MiB (16 instances) L3 缓存: 50 MiB (2 instances) NUMA 节点: 2 NUMA 节点0 CPU: 0-7,16-23 NUMA 节点1 CPU: 8-15,24-31 Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable Vulnerability Mds: Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable Vulnerability Meltdown: Mitigation; PTI Vulnerability Mmio stale data: Unknown: No mitigations Vulnerability Retbleed: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected ***CMake*** ***g++*** /usr/bin/g++ g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ***nvcc*** /usr/local/cuda/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Aug_15_22:02:13_PDT_2023 Cuda compilation tools, release 12.2, V12.2.140 Build cuda_12.2.r12.2/compiler.33191640_0 ***Python*** /home/wxl/anaconda3/bin/python Python 3.11.7 ***Environment Variables*** PATH : /usr/local/cuda/bin:/home/wxl/anaconda3/bin:/home/wxl/anaconda3/condabin:/home/wxl/.nvm/versions/node/v21.7.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin LD_LIBRARY_PATH : /usr/local/cuda/lib64 NUMBAPRO_NVVM : NUMBAPRO_LIBDEVICE : CONDA_PREFIX : /home/wxl/anaconda3 PYTHON_PATH : ***conda packages*** /home/wxl/anaconda3/bin/conda # packages in environment at /home/wxl/anaconda3: # # Name Version Build Channel _anaconda_depends 2024.02 py311_mkl_1 _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu abseil-cpp 20211102.0 hd4dd3e8_0 aiobotocore 2.7.0 py311h06a4308_0 aiohttp 3.9.3 py311h5eee18b_0 aioitertools 0.7.1 pyhd3eb1b0_0 aiosignal 1.2.0 pyhd3eb1b0_0 alabaster 0.7.12 pyhd3eb1b0_0 altair 5.0.1 py311h06a4308_0 anaconda-anon-usage 0.4.3 py311hfc0e8ea_100 anaconda-catalogs 0.2.0 py311h06a4308_0 anaconda-client 1.12.3 py311h06a4308_0 anaconda-cloud-auth 0.1.4 py311h06a4308_0 anaconda-navigator 2.5.2 py311h06a4308_0 anaconda-project 0.11.1 py311h06a4308_0 anyio 4.2.0 py311h06a4308_0 aom 3.6.0 h6a678d5_0 appdirs 1.4.4 pyhd3eb1b0_0 archspec 0.2.3 pyhd3eb1b0_0 argon2-cffi 21.3.0 pyhd3eb1b0_0 argon2-cffi-bindings 21.2.0 py311h5eee18b_0 arrow 1.2.3 py311h06a4308_1 arrow-cpp 14.0.2 h374c478_1 astroid 2.14.2 py311h06a4308_0 astropy 5.3.4 py311hf4808d0_0 asttokens 2.0.5 pyhd3eb1b0_0 async-lru 2.0.4 py311h06a4308_0 atomicwrites 1.4.0 py_0 attrs 23.1.0 py311h06a4308_0 automat 20.2.0 py_0 autopep8 1.6.0 pyhd3eb1b0_1 aws-c-auth 0.6.19 h5eee18b_0 aws-c-cal 0.5.20 hdbd6064_0 aws-c-common 0.8.5 h5eee18b_0 aws-c-compression 0.2.16 h5eee18b_0 aws-c-event-stream 0.2.15 h6a678d5_0 aws-c-http 0.6.25 h5eee18b_0 aws-c-io 0.13.10 h5eee18b_0 aws-c-mqtt 0.7.13 h5eee18b_0 aws-c-s3 0.1.51 hdbd6064_0 aws-c-sdkutils 0.1.6 h5eee18b_0 aws-checksums 0.1.13 h5eee18b_0 aws-crt-cpp 0.18.16 h6a678d5_0 aws-sdk-cpp 1.10.55 h721c034_0 babel 2.11.0 py311h06a4308_0 backports 1.1 pyhd3eb1b0_0 backports.functools_lru_cache 1.6.4 pyhd3eb1b0_0 backports.tempfile 1.0 pyhd3eb1b0_1 backports.weakref 1.0.post1 py_1 bcrypt 3.2.0 py311h5eee18b_1 beautifulsoup4 4.12.2 py311h06a4308_0 binaryornot 0.4.4 pyhd3eb1b0_1 black 23.11.0 py311h06a4308_0 blas 1.0 mkl bleach 4.1.0 pyhd3eb1b0_0 blinker 1.6.2 py311h06a4308_0 blosc 1.21.3 h6a678d5_0 bokeh 3.3.4 py311h92b7b1e_0 boltons 23.0.0 py311h06a4308_0 boost-cpp 1.82.0 hdb19cb5_2 botocore 1.31.64 py311h06a4308_0 bottleneck 1.3.7 py311hf4808d0_0 brotli 1.0.9 h5eee18b_7 brotli-bin 1.0.9 h5eee18b_7 brotli-python 1.0.9 py311h6a678d5_7 brunsli 0.1 h2531618_0 bs4 0.0.2 pypi_0 pypi bzip2 1.0.8 h7b6447c_0 c-ares 1.19.1 h5eee18b_0 c-blosc2 2.12.0 h80c7b02_0 ca-certificates 2024.3.11 h06a4308_0 cachetools 4.2.2 pyhd3eb1b0_0 certifi 2024.2.2 py311h06a4308_0 cffi 1.16.0 py311h5eee18b_0 cfitsio 3.470 h5893167_7 chardet 4.0.0 py311h06a4308_1003 charls 2.2.0 h2531618_0 charset-normalizer 2.0.4 pyhd3eb1b0_0 click 8.1.7 py311h06a4308_0 click-plugins 1.1.1 pypi_0 pypi cligj 0.7.2 pypi_0 pypi cloudpickle 2.2.1 py311h06a4308_0 clyent 1.2.2 py311h06a4308_1 colorama 0.4.6 py311h06a4308_0 colorcet 3.0.1 py311h06a4308_0 comm 0.1.2 py311h06a4308_0 conda 24.3.0 py311h06a4308_0 conda-build 24.1.2 py311h06a4308_0 conda-content-trust 0.2.0 py311h06a4308_0 conda-index 0.4.0 pyhd3eb1b0_0 conda-libmamba-solver 24.1.0 pyhd3eb1b0_0 conda-pack 0.6.0 pyhd3eb1b0_0 conda-package-handling 2.2.0 py311h06a4308_0 conda-package-streaming 0.9.0 py311h06a4308_0 conda-repo-cli 1.0.75 py311h06a4308_0 conda-token 0.4.0 pyhd3eb1b0_0 conda-verify 3.4.2 py_1 constantly 23.10.4 py311h06a4308_0 contourpy 1.2.0 py311hdb19cb5_0 cookiecutter 2.5.0 py311h06a4308_0 cryptography 42.0.2 py311hdda0065_0 cssselect 1.2.0 py311h06a4308_0 cuda-python 12.4.0 pypi_0 pypi cudf-cu12 24.4.0 pypi_0 pypi cugraph-cu12 24.4.0 pypi_0 pypi cuml-cu12 24.4.0 pypi_0 pypi cupy-cuda12x 13.0.0 pypi_0 pypi curl 8.5.0 hdbd6064_0 cuspatial-cu12 24.4.0 pypi_0 pypi cuvs-cu12 24.4.0 pypi_0 pypi cuxfilter-cu12 24.4.1 pypi_0 pypi cycler 0.11.0 pyhd3eb1b0_0 cyrus-sasl 2.1.28 h52b45da_1 cytoolz 0.12.2 py311h5eee18b_0 dask 2024.1.1 pypi_0 pypi dask-cuda 24.4.0 pypi_0 pypi dask-cudf-cu12 24.4.0 pypi_0 pypi dask-expr 0.4.0 pypi_0 pypi datashader 0.16.0 py311h06a4308_0 dav1d 1.2.1 h5eee18b_0 dbus 1.13.18 hb2f20db_0 debugpy 1.6.7 py311h6a678d5_0 decorator 5.1.1 pyhd3eb1b0_0 defusedxml 0.7.1 pyhd3eb1b0_0 diff-match-patch 20200713 pyhd3eb1b0_0 dill 0.3.7 py311h06a4308_0 distributed 2024.1.1 pypi_0 pypi distro 1.8.0 py311h06a4308_0 docstring-to-markdown 0.11 py311h06a4308_0 docutils 0.18.1 py311h06a4308_3 entrypoints 0.4 py311h06a4308_0 et_xmlfile 1.1.0 py311h06a4308_0 executing 0.8.3 pyhd3eb1b0_0 expat 2.5.0 h6a678d5_0 fastrlock 0.8.2 pypi_0 pypi filelock 3.13.1 py311h06a4308_0 fiona 1.9.6 pypi_0 pypi flake8 6.0.0 py311h06a4308_0 flask 2.2.5 py311h06a4308_0 fmt 9.1.0 hdb19cb5_0 fontconfig 2.14.1 h4c34cd2_2 fonttools 4.25.0 pyhd3eb1b0_0 fqdn 1.5.1 pypi_0 pypi freetype 2.12.1 h4a9f257_0 frozenlist 1.4.0 py311h5eee18b_0 fsspec 2023.10.0 py311h06a4308_0 future 0.18.3 py311h06a4308_0 gensim 4.3.0 py311hba01205_1 geopandas 0.14.3 pypi_0 pypi gflags 2.2.2 h6a678d5_1 giflib 5.2.1 h5eee18b_3 gitdb 4.0.7 pyhd3eb1b0_0 gitpython 3.1.37 py311h06a4308_0 glib 2.78.4 h6a678d5_0 glib-tools 2.78.4 h6a678d5_0 glog 0.5.0 h6a678d5_1 gmp 6.2.1 h295c915_3 gmpy2 2.1.2 py311hc9b5ff0_0 gpudtw 0.13 pypi_0 pypi greenlet 3.0.1 py311h6a678d5_0 grpc-cpp 1.48.2 he1ff14a_1 gst-plugins-base 1.14.1 h6a678d5_1 gstreamer 1.14.1 h5eee18b_1 h5py 3.9.0 py311hdd6beaf_0 hdf5 1.12.1 h2b7332f_3 heapdict 1.0.1 pyhd3eb1b0_0 holoviews 1.18.3 py311h06a4308_0 hvplot 0.9.2 py311h06a4308_0 hyperlink 21.0.0 pyhd3eb1b0_0 icu 73.1 h6a678d5_0 idna 3.4 py311h06a4308_0 imagecodecs 2023.1.23 py311h8105a5c_0 imageio 2.33.1 py311h06a4308_0 imagesize 1.4.1 py311h06a4308_0 imbalanced-learn 0.11.0 py311h06a4308_1 importlib-metadata 7.0.1 py311h06a4308_0 importlib_metadata 7.0.1 hd3eb1b0_0 incremental 22.10.0 pyhd3eb1b0_0 inflection 0.5.1 py311h06a4308_0 iniconfig 1.1.1 pyhd3eb1b0_0 intake 0.6.8 py311h06a4308_0 intel-openmp 2023.1.0 hdb19cb5_46306 intervaltree 3.1.0 pyhd3eb1b0_0 ipykernel 6.28.0 py311h06a4308_0 ipython 8.20.0 py311h06a4308_0 ipython_genutils 0.2.0 pyhd3eb1b0_1 ipywidgets 7.6.5 pyhd3eb1b0_2 isoduration 20.11.0 pypi_0 pypi isort 5.9.3 pyhd3eb1b0_0 itemadapter 0.3.0 pyhd3eb1b0_0 itemloaders 1.1.0 py311h06a4308_0 itsdangerous 2.0.1 pyhd3eb1b0_0 jaraco.classes 3.2.1 pyhd3eb1b0_0 jedi 0.18.1 py311h06a4308_1 jeepney 0.7.1 pyhd3eb1b0_0 jellyfish 1.0.1 py311hb02cf49_0 jinja2 3.1.3 py311h06a4308_0 jmespath 1.0.1 py311h06a4308_0 joblib 1.2.0 py311h06a4308_0 jpeg 9e h5eee18b_1 jq 1.6 h27cfd23_1000 json5 0.9.6 pyhd3eb1b0_0 jsonpatch 1.32 pyhd3eb1b0_0 jsonpointer 2.1 pyhd3eb1b0_0 jsonschema 4.19.2 py311h06a4308_0 jsonschema-specifications 2023.7.1 py311h06a4308_0 jupyter 1.0.0 py311h06a4308_9 jupyter-lsp 2.2.0 py311h06a4308_0 jupyter-server-proxy 4.1.2 pypi_0 pypi jupyter_client 8.6.0 py311h06a4308_0 jupyter_console 6.6.3 py311h06a4308_0 jupyter_core 5.5.0 py311h06a4308_0 jupyter_events 0.8.0 py311h06a4308_0 jupyter_server 2.10.0 py311h06a4308_0 jupyter_server_terminals 0.4.4 py311h06a4308_1 jupyterlab 4.0.11 py311h06a4308_0 jupyterlab-variableinspector 3.1.0 py311h06a4308_0 jupyterlab_pygments 0.1.2 py_0 jupyterlab_server 2.25.1 py311h06a4308_0 jupyterlab_widgets 3.0.9 py311h06a4308_0 jxrlib 1.1 h7b6447c_2 keyring 23.13.1 py311h06a4308_0 kiwisolver 1.4.4 py311h6a678d5_0 krb5 1.20.1 h143b758_1 lazy-object-proxy 1.6.0 py311h5eee18b_0 lazy_loader 0.3 py311h06a4308_0 lcms2 2.12 h3be6417_0 ld_impl_linux-64 2.38 h1181459_1 lerc 3.0 h295c915_0 libaec 1.0.4 he6710b0_1 libarchive 3.6.2 h6ac8c49_2 libavif 0.11.1 h5eee18b_0 libboost 1.82.0 h109eef0_2 libbrotlicommon 1.0.9 h5eee18b_7 libbrotlidec 1.0.9 h5eee18b_7 libbrotlienc 1.0.9 h5eee18b_7 libclang 14.0.6 default_hc6dbbc7_1 libclang13 14.0.6 default_he11475f_1 libcups 2.4.2 h2d74bed_1 libcurl 8.5.0 h251f7ec_0 libdeflate 1.17 h5eee18b_1 libedit 3.1.20230828 h5eee18b_0 libev 4.33 h7f8727e_1 libevent 2.1.12 hdbd6064_1 libffi 3.4.4 h6a678d5_0 libgcc-ng 11.2.0 h1234567_1 libgfortran-ng 11.2.0 h00389a5_1 libgfortran5 11.2.0 h1234567_1 libglib 2.78.4 hdc74915_0 libgomp 11.2.0 h1234567_1 libiconv 1.16 h7f8727e_2 liblief 0.12.3 h6a678d5_0 libllvm14 14.0.6 hdb19cb5_3 libmamba 1.5.6 haf1ee3a_0 libmambapy 1.5.6 py311h2dafd23_0 libnghttp2 1.57.0 h2d74bed_0 libpng 1.6.39 h5eee18b_0 libpq 12.17 hdbd6064_0 libprotobuf 3.20.3 he621ea3_0 libsodium 1.0.18 h7b6447c_0 libsolv 0.7.24 he621ea3_0 libspatialindex 1.9.3 h2531618_0 libssh2 1.10.0 hdbd6064_2 libstdcxx-ng 11.2.0 h1234567_1 libthrift 0.15.0 h1795dd8_2 libtiff 4.5.1 h6a678d5_0 libuuid 1.41.5 h5eee18b_0 libwebp-base 1.3.2 h5eee18b_0 libxcb 1.15 h7f8727e_0 libxkbcommon 1.0.1 h5eee18b_1 libxml2 2.10.4 hf1b16e4_1 libxslt 1.1.37 h5eee18b_1 libzopfli 1.0.3 he6710b0_0 linkify-it-py 2.0.0 py311h06a4308_0 llvmlite 0.42.0 py311h6a678d5_0 locket 1.0.0 py311h06a4308_0 lxml 4.9.3 py311hdbbb534_0 lz4 4.3.2 py311h5eee18b_0 lz4-c 1.9.4 h6a678d5_0 lzo 2.10 h7b6447c_2 mako 1.3.3 pypi_0 pypi markdown 3.4.1 py311h06a4308_0 markdown-it-py 2.2.0 py311h06a4308_1 markupsafe 2.1.3 py311h5eee18b_0 matplotlib 3.8.0 py311h06a4308_0 matplotlib-base 3.8.0 py311ha02d727_0 matplotlib-inline 0.1.6 py311h06a4308_0 mccabe 0.7.0 pyhd3eb1b0_0 mdit-py-plugins 0.3.0 py311h06a4308_0 mdurl 0.1.0 py311h06a4308_0 menuinst 2.0.2 py311h06a4308_0 mistune 2.0.4 py311h06a4308_0 mkl 2023.1.0 h213fc3f_46344 mkl-service 2.4.0 py311h5eee18b_1 mkl_fft 1.3.8 py311h5eee18b_0 mkl_random 1.2.4 py311hdb19cb5_0 modin 0.29.0 pypi_0 pypi more-itertools 10.1.0 py311h06a4308_0 mpc 1.1.0 h10f8cd9_1 mpfr 4.0.2 hb69a4c5_1 mpmath 1.3.0 py311h06a4308_0 msgpack 1.0.8 pypi_0 pypi multidict 6.0.4 py311h5eee18b_0 multipledispatch 0.6.0 py311h06a4308_0 munkres 1.1.4 py_0 mypy 1.8.0 py311h5eee18b_0 mypy_extensions 1.0.0 py311h06a4308_0 mysql 5.7.24 h721c034_2 navigator-updater 0.4.0 py311h06a4308_1 nbclient 0.8.0 py311h06a4308_0 nbconvert 7.10.0 py311h06a4308_0 nbformat 5.9.2 py311h06a4308_0 ncurses 6.4 h6a678d5_0 nest-asyncio 1.6.0 py311h06a4308_0 networkx 3.1 py311h06a4308_0 nltk 3.8.1 py311h06a4308_0 notebook 7.0.8 py311h06a4308_0 notebook-shim 0.2.3 py311h06a4308_0 nspr 4.35 h6a678d5_0 nss 3.89.1 h6a678d5_0 numba 0.59.0 py311ha02d727_0 numexpr 2.8.7 py311h65dcdc2_0 numpy 1.26.4 py311h08b1b3b_0 numpy-base 1.26.4 py311hf175353_0 numpydoc 1.5.0 py311h06a4308_0 nvtx 0.2.10 pypi_0 pypi oniguruma 6.9.7.1 h27cfd23_0 openjpeg 2.4.0 h3ad879b_0 openpyxl 3.0.10 py311h5eee18b_0 openssl 3.0.13 h7f8727e_0 orc 1.7.4 hb3bc3d3_1 overrides 7.4.0 py311h06a4308_0 packaging 23.1 py311h06a4308_0 pandas 2.2.1 pypi_0 pypi pandocfilters 1.5.0 pyhd3eb1b0_0 panel 1.3.8 py311h06a4308_0 param 2.0.2 py311h06a4308_0 parsel 1.8.1 py311h06a4308_0 parso 0.8.3 pyhd3eb1b0_0 partd 1.4.1 py311h06a4308_0 patch 2.7.6 h7b6447c_1001 patchelf 0.17.2 h6a678d5_0 pathlib 1.0.1 pyhd3eb1b0_1 pathspec 0.10.3 py311h06a4308_0 patsy 0.5.3 py311h06a4308_0 pcre2 10.42 hebb0a14_0 pexpect 4.8.0 pyhd3eb1b0_3 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 10.2.0 py311h5eee18b_0 pip 23.3.1 py311h06a4308_0 pkce 1.0.3 py311h06a4308_0 pkginfo 1.9.6 py311h06a4308_0 platformdirs 3.10.0 py311h06a4308_0 plotly 5.9.0 py311h06a4308_0 pluggy 1.0.0 py311h06a4308_1 ply 3.11 py311h06a4308_0 polars 0.20.22 pypi_0 pypi polars-lts-cpu 0.20.22 pypi_0 pypi prometheus_client 0.14.1 py311h06a4308_0 prompt-toolkit 3.0.43 py311h06a4308_0 prompt_toolkit 3.0.43 hd3eb1b0_0 protego 0.1.16 py_0 protobuf 4.25.3 pypi_0 pypi psutil 5.9.0 py311h5eee18b_0 psycopg2 2.9.9 pypi_0 pypi ptyprocess 0.7.0 pyhd3eb1b0_2 pure_eval 0.2.2 pyhd3eb1b0_0 py-cpuinfo 9.0.0 py311h06a4308_0 py-lief 0.12.3 py311h6a678d5_0 pyarrow 14.0.2 py311hb6e97c4_0 pyasn1 0.4.8 pyhd3eb1b0_0 pyasn1-modules 0.2.8 py_0 pybind11-abi 4 hd3eb1b0_1 pycodestyle 2.10.0 py311h06a4308_0 pycosat 0.6.6 py311h5eee18b_0 pycparser 2.21 pyhd3eb1b0_0 pyct 0.5.0 py311h06a4308_0 pycuda 2024.1 pypi_0 pypi pycurl 7.45.2 py311hdbd6064_1 pydantic 1.10.12 py311h5eee18b_1 pydeck 0.8.0 py311h06a4308_2 pydispatcher 2.0.5 py311h06a4308_2 pydocstyle 6.3.0 py311h06a4308_0 pyerfa 2.0.0 py311h5eee18b_0 pyflakes 3.0.1 py311h06a4308_0 pygments 2.15.1 py311h06a4308_1 pyjwt 2.4.0 py311h06a4308_0 pylibcugraph-cu12 24.4.0 pypi_0 pypi pylibraft-cu12 24.4.0 pypi_0 pypi pylint 2.16.2 py311h06a4308_0 pylint-venv 2.3.0 py311h06a4308_0 pyls-spyder 0.4.0 pyhd3eb1b0_0 pynvjitlink-cu12 0.1.14 pypi_0 pypi pynvml 11.4.1 pypi_0 pypi pyodbc 5.0.1 py311h6a678d5_0 pyopencl 2024.1 pypi_0 pypi pyopenssl 24.0.0 py311h06a4308_0 pyparsing 3.0.9 py311h06a4308_0 pyproj 3.6.1 pypi_0 pypi pyqt 5.15.10 py311h6a678d5_0 pyqt5-sip 12.13.0 py311h5eee18b_0 pyqtwebengine 5.15.10 py311h6a678d5_0 pysocks 1.7.1 py311h06a4308_0 pytables 3.9.2 py311h9d13977_0 pytest 7.4.0 py311h06a4308_0 python 3.11.7 h955ad1f_0 python-dateutil 2.8.2 pyhd3eb1b0_0 python-dotenv 0.21.0 py311h06a4308_0 python-fastjsonschema 2.16.2 py311h06a4308_0 python-json-logger 2.0.7 py311h06a4308_0 python-libarchive-c 2.9 pyhd3eb1b0_1 python-lmdb 1.4.1 py311h6a678d5_0 python-lsp-black 1.2.1 py311h06a4308_0 python-lsp-jsonrpc 1.0.0 pyhd3eb1b0_0 python-lsp-server 1.7.2 py311h06a4308_0 python-slugify 5.0.2 pyhd3eb1b0_0 python-snappy 0.6.1 py311h6a678d5_0 python-tzdata 2023.3 pyhd3eb1b0_0 pytoolconfig 1.2.6 py311h06a4308_0 pytools 2024.1.1 pypi_0 pypi pytz 2023.3.post1 py311h06a4308_0 pyviz_comms 3.0.0 py311h06a4308_0 pywavelets 1.5.0 py311hf4808d0_0 pyxdg 0.27 pyhd3eb1b0_0 pyyaml 6.0.1 py311h5eee18b_0 pyzmq 25.1.2 py311h6a678d5_0 qdarkstyle 3.0.2 pyhd3eb1b0_0 qstylizer 0.2.2 py311h06a4308_0 qt-main 5.15.2 h53bd1ea_10 qt-webengine 5.15.9 h9ab4d14_7 qtawesome 1.2.2 py311h06a4308_0 qtconsole 5.4.2 py311h06a4308_0 qtpy 2.4.1 py311h06a4308_0 queuelib 1.6.2 py311h06a4308_0 raft-dask-cu12 24.4.0 pypi_0 pypi rapids-dask-dependency 24.4.1 pypi_0 pypi re2 2022.04.01 h295c915_0 readline 8.2 h5eee18b_0 referencing 0.30.2 py311h06a4308_0 regex 2023.10.3 py311h5eee18b_0 reproc 14.2.4 h295c915_1 reproc-cpp 14.2.4 h295c915_1 requests 2.31.0 py311h06a4308_1 requests-file 1.5.1 pyhd3eb1b0_0 requests-toolbelt 1.0.0 py311h06a4308_0 rfc3339-validator 0.1.4 py311h06a4308_0 rfc3986-validator 0.1.1 py311h06a4308_0 rich 13.3.5 py311h06a4308_0 rmm-cu12 24.4.0 pypi_0 pypi rope 1.7.0 py311h06a4308_0 rpds-py 0.10.6 py311hb02cf49_0 rtree 1.0.1 py311h06a4308_0 ruamel.yaml 0.17.21 py311h5eee18b_0 ruamel_yaml 0.17.21 py311h5eee18b_0 s2n 1.3.27 hdbd6064_0 s3fs 2023.10.0 py311h06a4308_0 scikit-image 0.22.0 py311ha02d727_0 scikit-learn 1.2.2 py311h6a678d5_1 scipy 1.11.4 py311h08b1b3b_0 scrapy 2.8.0 py311h06a4308_0 seaborn 0.12.2 py311h06a4308_0 secretstorage 3.3.1 py311h06a4308_1 semver 2.13.0 pyhd3eb1b0_0 send2trash 1.8.2 py311h06a4308_0 service_identity 18.1.0 pyhd3eb1b0_1 setuptools 68.2.2 py311h06a4308_0 shapely 2.0.3 pypi_0 pypi simpervisor 1.0.0 pypi_0 pypi simplejson 3.19.2 pypi_0 pypi sip 6.7.12 py311h6a678d5_0 six 1.16.0 pyhd3eb1b0_1 smart_open 5.2.1 py311h06a4308_0 smmap 4.0.0 pyhd3eb1b0_0 snappy 1.1.10 h6a678d5_1 sniffio 1.3.0 py311h06a4308_0 snowballstemmer 2.2.0 pyhd3eb1b0_0 sortedcontainers 2.4.0 pyhd3eb1b0_0 soupsieve 2.5 py311h06a4308_0 sphinx 5.0.2 py311h06a4308_0 sphinxcontrib-applehelp 1.0.2 pyhd3eb1b0_0 sphinxcontrib-devhelp 1.0.2 pyhd3eb1b0_0 sphinxcontrib-htmlhelp 2.0.0 pyhd3eb1b0_0 sphinxcontrib-jsmath 1.0.1 pyhd3eb1b0_0 sphinxcontrib-qthelp 1.0.3 pyhd3eb1b0_0 sphinxcontrib-serializinghtml 1.1.5 pyhd3eb1b0_0 spyder 5.4.3 py311h06a4308_1 spyder-kernels 2.4.4 py311h06a4308_0 sqlalchemy 2.0.25 py311h5eee18b_0 sqlite 3.41.2 h5eee18b_0 stack_data 0.2.0 pyhd3eb1b0_0 statsmodels 0.14.0 py311hf4808d0_0 streamlit 1.30.0 py311h06a4308_0 sympy 1.12 py311h06a4308_0 tabulate 0.9.0 py311h06a4308_0 tbb 2021.8.0 hdb19cb5_0 tblib 1.7.0 pyhd3eb1b0_0 tenacity 8.2.2 py311h06a4308_0 terminado 0.17.1 py311h06a4308_0 text-unidecode 1.3 pyhd3eb1b0_0 textdistance 4.2.1 pyhd3eb1b0_0 threadpoolctl 2.2.0 pyh0d69192_0 three-merge 0.1.1 pyhd3eb1b0_0 tifffile 2023.4.12 py311h06a4308_0 tinycss2 1.2.1 py311h06a4308_0 tk 8.6.12 h1ccaba5_0 tldextract 3.2.0 pyhd3eb1b0_0 toml 0.10.2 pyhd3eb1b0_0 tomlkit 0.11.1 py311h06a4308_0 toolz 0.12.0 py311h06a4308_0 tornado 6.3.3 py311h5eee18b_0 tqdm 4.65.0 py311h92b7b1e_0 traitlets 5.7.1 py311h06a4308_0 treelite 4.1.2 pypi_0 pypi truststore 0.8.0 py311h06a4308_0 tslearn 0.6.3 pypi_0 pypi tushare 1.4.6 pypi_0 pypi twisted 23.10.0 py311h06a4308_0 typing-extensions 4.9.0 py311h06a4308_1 typing_extensions 4.9.0 py311h06a4308_1 tzdata 2023d h04d1e81_0 tzlocal 2.1 py311h06a4308_1 uc-micro-py 1.0.1 py311h06a4308_0 ucx-py-cu12 0.37.0 pypi_0 pypi ujson 5.4.0 py311h6a678d5_0 unidecode 1.2.0 pyhd3eb1b0_0 unixodbc 2.3.11 h5eee18b_0 uri-template 1.3.0 pypi_0 pypi urllib3 2.0.7 py311h06a4308_0 utf8proc 2.6.1 h5eee18b_1 validators 0.18.2 pyhd3eb1b0_0 w3lib 2.1.2 py311h06a4308_0 watchdog 2.1.6 py311h06a4308_0 wcwidth 0.2.5 pyhd3eb1b0_0 webcolors 1.13 pypi_0 pypi webencodings 0.5.1 py311h06a4308_1 websocket-client 0.57.0 pypi_0 pypi werkzeug 2.2.3 py311h06a4308_0 whatthepatch 1.0.2 py311h06a4308_0 wheel 0.41.2 py311h06a4308_0 widgetsnbextension 3.5.2 py311h06a4308_1 wrapt 1.14.1 py311h5eee18b_0 wurlitzer 3.0.2 py311h06a4308_0 xarray 2023.6.0 py311h06a4308_0 xyzservices 2022.9.0 py311h06a4308_1 xz 5.4.5 h5eee18b_0 yaml 0.2.5 h7b6447c_0 yaml-cpp 0.8.0 h6a678d5_0 yapf 0.31.0 pyhd3eb1b0_0 yarl 1.9.3 py311h5eee18b_0 zeromq 4.3.5 h6a678d5_0 zfp 1.0.0 h6a678d5_0 zict 3.0.0 py311h06a4308_0 zipp 3.17.0 py311h06a4308_0 zlib 1.2.13 h5eee18b_0 zlib-ng 2.0.7 h5eee18b_0 zope 1.0 py311h06a4308_1 zope.interface 5.4.0 py311h5eee18b_0 zstandard 0.19.0 py311h5eee18b_0 zstd 1.5.5 hc292b87_0
@wuxianliang I also have issues with dask NN. And I run into the same error code as you.