Kernel crashing while executing convert_catboost

Question

Kernel crashing while executing convert_catboost

hanzigs opened this issue a year ago · comments

My python kernel dies in VS Code whenever I run convert_catboost,
this is not happening for convert_sklearn, convert_xgboost, convert_lightgbm, also not for tf2onnx.utils.save_onnx_model,
only happening for

from onnxmltools.convert.common.data_types import FloatTensorType
from onnxmltools.convert import convert_catboost
input_shape = explainer_columns.shape[0]
initial_type = [('float_input', FloatTensorType([1, input_shape]))]
onnx_model = convert_catboost(catboostmodel, initial_types=initial_type)

and end up in

below is requirements file on Python 3.10.9 environment

absl-py==1.4.0
asttokens==2.2.1
astunparse==1.6.3
autokeras==1.0.20
backcall==0.2.0
cachetools==5.3.0
catboost==1.1.1
certifi==2022.12.7
charset-normalizer==3.1.0
click==8.1.3
colorama==0.4.6
coloredlogs==15.0.1
combo==0.1.3
comm==0.1.3
contourpy==1.0.7
cycler==0.11.0
debugpy==1.6.7
decorator==5.1.1
dill==0.3.6
elasticsearch==7.12.0
et-xmlfile==1.1.0
executing==1.2.0
Flask==2.2.3
flatbuffers==2.0.7
fonttools==4.39.3
gast==0.4.0
google-auth==2.17.3
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
graphviz==0.20.1
grpcio==1.53.0
h5py==3.8.0
humanfriendly==10.0
idna==3.4
imageio==2.27.0
imbalanced-learn==0.10.1
ipykernel==6.22.0
ipython==8.12.0
ipython-genutils==0.2.0
itsdangerous==2.1.2
jax==0.4.8
jedi==0.18.2
Jinja2==3.1.2
joblib==1.2.0
jupyter_client==8.2.0
jupyter_core==5.3.0
keras==2.12.0
keras-tuner==1.3.0
kiwisolver==1.4.4
kneed==0.8.2
kt-legacy==1.0.5
lazy_loader==0.2
libclang==16.0.0
lightgbm==3.3.5
lime==0.2.0.1
llvmlite==0.39.1
Markdown==3.4.3
MarkupSafe==2.1.2
matplotlib==3.7.1
matplotlib-inline==0.1.6
ml-dtypes==0.1.0
mpmath==1.3.0
nest-asyncio==1.5.6
networkx==3.1
niapy==2.0.5
numba==0.56.4
numpy==1.23.5
oauthlib==3.2.2
onnx==1.13.1
onnxconverter-common==1.13.0
onnxmltools==1.11.2
onnxruntime==1.14.1
openpyxl==3.1.2
opt-einsum==3.3.0
packaging==23.1
pandas==1.5.3
parso==0.8.3
pickleshare==0.7.5
Pillow==9.5.0
platformdirs==3.2.0
plotly==5.14.1
prompt-toolkit==3.0.38
protobuf==3.20.3
psutil==5.9.4
pure-eval==0.2.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.15.0
pyod==1.0.8
pyparsing==3.0.9
pyreadline3==3.4.1
python-dateutil==2.8.2
pytz==2023.3
PyWavelets==1.4.1
pywin32==306
pyzmq==25.0.2
requests==2.28.2
requests-oauthlib==1.3.1
rsa==4.9
scikit-image==0.20.0
scikit-learn==1.2.2
scipy==1.10.1
seaborn==0.12.2
six==1.16.0
skl2onnx==1.14.0
sklearn-nature-inspired-algorithms==0.11.0
stack-data==0.6.2
sympy==1.11.1
tenacity==8.2.2
tensorboard==2.12.2
tensorboard-data-server==0.7.0
tensorboard-plugin-wit==1.8.1
tensorflow==2.12.0
tensorflow-estimator==2.12.0
tensorflow-intel==2.12.0
tensorflow-io-gcs-filesystem==0.31.0
termcolor==2.2.0
tf2onnx==1.13.0
threadpoolctl==3.1.0
tifffile==2023.4.12
toml==0.10.2
tornado==6.2
tqdm==4.65.0
traitlets==5.9.0
typing_extensions==4.5.0
urllib3==1.26.15
wcwidth==0.2.6
Werkzeug==2.2.3
wincertstore==0.2
wrapt==1.14.1
xgboost==1.7.4

Below is the Jupyter log

Visual Studio Code (1.77.3, undefined, desktop)
Jupyter Extension Version: 2023.3.1201040234.
Python Extension Version: 2023.6.0.
Workspace folder c:\PG\AL\ALpy3109
User belongs to experiment group 'jupyterEnhancedDataViewer'
User belongs to experiment group 'jupyterTestcf'
info 13:46:07.091: LSP Notebooks experiment is enabled
warn 13:46:07.581: Failed to get activated env vars for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe in 384ms
info 13:46:07.599: Process Execution: > ~\AppData\Local\Programs\Python\Python39\python.exe -c "import site;print("USER_BASE_VALUE");print(site.USER_SITE);print("USER_BASE_VALUE");"
> ~\AppData\Local\Programs\Python\Python39\python.exe -c "import site;print("USER_BASE_VALUE");print(site.USER_SITE);print("USER_BASE_VALUE");"
info 13:46:07.715: Process Execution: > ~\AppData\Local\Programs\Python\Python39\python.exe -m pip list
> ~\AppData\Local\Programs\Python\Python39\python.exe -m pip list
info 13:46:55.741: Starting interactive window for resource 'c:\PG\AL\ALpy3109\TestAL.py' with controller '.jvsc74a57bd02e87716c8bea2ac851efcdaa01e56835bafbb5876c83923fb1949cd8a7b7ea5a.c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe.c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe.-m#ipykernel_launcher (Interactive)'
info 13:46:56.180: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd02e87716c8bea2ac851efcdaa01e56835bafbb5876c83923fb1949cd8a7b7ea5a.c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe.c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe.-m#ipykernel_launcher (Python Path: , EnvType: Unknown, EnvName: '', Version: 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]) for 'Interactive-1.interactive' (disableUI=false)
warn 13:46:56.359: Failed to get activated env vars for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe in 175ms
info 13:46:56.366: Process Execution: > ~\AppData\Local\Programs\Python\Python39\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> ~\AppData\Local\Programs\Python\Python39\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
warn 13:46:56.369: Failed to get activated env vars for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe in 84ms
warn 13:46:56.371: Failed to get activated env vars for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe in 85ms
warn 13:46:56.390: Failed to get activated env vars for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe in 16ms
info 13:46:56.397: Process Execution: > ~\AppData\Local\Programs\Python\Python39\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\vscode_datascience_helpers\kernel_interrupt_daemon.py --ppid 19760
> ~\AppData\Local\Programs\Python\Python39\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\vscode_datascience_helpers\kernel_interrupt_daemon.py --ppid 19760
info 13:46:56.397: Process Execution: cwd: ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\vscode_datascience_helpers
cwd: ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\vscode_datascience_helpers
info 13:46:56.538: Process Execution: > ~\AppData\Local\Programs\Python\Python39\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"16dbdbbb-8bce-43f8-9788-4c612090e50a" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-19760opM0Lcv0sd8t.json
> ~\AppData\Local\Programs\Python\Python39\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"16dbdbbb-8bce-43f8-9788-4c612090e50a" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-19760opM0Lcv0sd8t.json
info 13:46:56.539: Process Execution: cwd: c:\PG\AL\ALpy3109
cwd: c:\PG\AL\ALpy3109
info 13:46:56.952: ipykernel version & path 6.5.0, ~\AppData\Local\Programs\Python\Python39\lib\site-packages\ipykernel\__init__.py for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe
warn 13:46:57.787: StdErr from Kernel Process c:\Users\PG\AppData\Local\Programs\Python\Python39\lib\site-packages\traitlets\traitlets.py:2202: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn(

warn 13:46:57.787: StdErr from Kernel Process c:\Users\PG\AppData\Local\Programs\Python\Python39\lib\site-packages\traitlets\traitlets.py:2157: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use '16dbdbbb-8bce-43f8-9788-4c612090e50a' instead of 'b"16dbdbbb-8bce-43f8-9788-4c612090e50a"'.
  warn(

info 13:46:57.998: Started Kernel Python 3.9.7 (pid: 1408)
info 13:47:00.131: Generated code for 1 = <ipython-input-1-e0bc212ab7a0> with 1 lines
warn 13:47:00.196: Failed to get activated env vars for c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe in 2147ms
info 13:47:00.204: Process Execution: > ~\AppData\Local\Programs\Python\Python39\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\printJupyterDataDir.py
> ~\AppData\Local\Programs\Python\Python39\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\printJupyterDataDir.py
info 13:47:13.427: Process Execution: > ~\anaconda3\envs\mlpy3109_t6\python.exe -m pip list
> ~\anaconda3\envs\mlpy3109_t6\python.exe -m pip list
info 13:47:18.900: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -m pip list
> ~\anaconda3\envs\ALpy3109_1\python.exe -m pip list
info 13:47:28.492: Disposing kernel .jvsc74a57bd02e87716c8bea2ac851efcdaa01e56835bafbb5876c83923fb1949cd8a7b7ea5a.c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe.c:\Users\PG\AppData\Local\Programs\Python\Python39\python.exe.-m#ipykernel_launcher for notebook Interactive-1.interactive due to selection of another kernel or closing of the notebook
info 13:47:28.492: Dispose Kernel 'Interactive-1.interactive' associated with 'c:\PG\AL\ALpy3109\TestAL.py'
info 13:47:28.494: Dispose Kernel process 1408.
info 13:47:28.518: Dispose Kernel 'Interactive-1.interactive' associated with 'c:\PG\AL\ALpy3109\TestAL.py'
info 13:47:28.524: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd0228f8367c5fae163835e96f80a514892220503e16a5d9c5b64567d9fe886671e.c:\Users\PG\anaconda3\envs\ALpy3109_1\python.exe.c:\Users\PG\anaconda3\envs\ALpy3109_1\python.exe.-m#ipykernel_launcher (Python Path: c:\Users\PG\anaconda3\envs\ALpy3109_1, EnvType: Conda, EnvName: 'ALpy3109_1', Version: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar  8 2023, 10:42:25) [MSC v.1916 64 bit (AMD64)]) for 'Interactive-1.interactive' (disableUI=false)
info 13:47:28.554: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> ~\anaconda3\envs\ALpy3109_1\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 13:47:28.684: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9008 --control=9006 --hb=9005 --Session.signature_scheme="hmac-sha256" --Session.key=b"1e501472-2471-4691-ac7a-0fdd2951c801" --shell=9007 --transport="tcp" --iopub=9009 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-19760Ko041hg24tQe.json
> ~\anaconda3\envs\ALpy3109_1\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9008 --control=9006 --hb=9005 --Session.signature_scheme="hmac-sha256" --Session.key=b"1e501472-2471-4691-ac7a-0fdd2951c801" --shell=9007 --transport="tcp" --iopub=9009 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-19760Ko041hg24tQe.json
info 13:47:28.684: Process Execution: cwd: c:\PG\AL\ALpy3109
cwd: c:\PG\AL\ALpy3109
info 13:47:29.294: ipykernel version & path 6.22.0, ~\anaconda3\envs\ALpy3109_1\lib\site-packages\ipykernel\__init__.py for c:\Users\PG\anaconda3\envs\ALpy3109_1\python.exe
info 13:47:30.751: Started Kernel ALpy3109_1 (Python 3.10.9) (pid: 9076)
info 13:47:30.829: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\printJupyterDataDir.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\printJupyterDataDir.py
info 13:47:33.324: Restart kernel execution
info 13:47:33.324: Restart requested vscode-interactive:/Interactive-1.interactive
info 13:47:33.325: Restarting 03c93d32-3de6-46c2-81a6-04a86f818ec3
info 13:47:33.333: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> ~\anaconda3\envs\ALpy3109_1\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 13:47:33.398: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"652331f5-b39b-4ca7-9baf-81ff16ce827c" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-197601agHAFU5fPkU.json
> ~\anaconda3\envs\ALpy3109_1\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"652331f5-b39b-4ca7-9baf-81ff16ce827c" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-197601agHAFU5fPkU.json
info 13:47:33.398: Process Execution: cwd: c:\PG\AL\ALpy3109
cwd: c:\PG\AL\ALpy3109
info 13:47:33.836: ipykernel version & path 6.22.0, ~\anaconda3\envs\ALpy3109_1\lib\site-packages\ipykernel\__init__.py for c:\Users\PG\anaconda3\envs\ALpy3109_1\python.exe
info 13:47:35.058: Got new session 9342fed1-ce2d-463d-961f-4300cfd4fa65
info 13:47:35.058: Started new restart session
info 13:47:35.058: Dispose Kernel process 9076.
info 13:47:37.941: Generated code for 1 = <ipython-input-1-e0bc212ab7a0> with 1 lines
info 13:47:40.604: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:47:40.741: Generated code for 2 = <ipython-input-2-087efb3b7db3> with 7 lines
info 13:47:47.120: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:47:47.218: Generated code for 3 = <ipython-input-3-4db6906727a9> with 20 lines
info 13:47:50.503: Generated code for 4 = <ipython-input-4-fa2cd79fb24e> with 1 lines
info 13:47:52.086: Generated code for 5 = <ipython-input-5-507001a65ffe> with 1 lines
info 13:48:03.938: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:48:04.036: Generated code for 6 = <ipython-input-6-041e1b82d363> with 68 lines
info 13:48:22.697: Cancel all remaining cells true || Error || undefined
info 13:48:35.588: Restart kernel execution
info 13:48:35.589: Restart requested vscode-interactive:/Interactive-1.interactive
info 13:48:35.589: Restarting 9342fed1-ce2d-463d-961f-4300cfd4fa65
info 13:48:35.596: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> ~\anaconda3\envs\ALpy3109_1\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 13:48:35.695: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9008 --control=9006 --hb=9005 --Session.signature_scheme="hmac-sha256" --Session.key=b"3ff8d1b6-178e-402f-b831-0d24a86dd3cf" --shell=9007 --transport="tcp" --iopub=9009 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-19760YWw0U3MPd2w8.json
> ~\anaconda3\envs\ALpy3109_1\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9008 --control=9006 --hb=9005 --Session.signature_scheme="hmac-sha256" --Session.key=b"3ff8d1b6-178e-402f-b831-0d24a86dd3cf" --shell=9007 --transport="tcp" --iopub=9009 --f=c:\Users\PG\AppData\Roaming\jupyter\runtime\kernel-v2-19760YWw0U3MPd2w8.json
info 13:48:35.695: Process Execution: cwd: c:\PG\AL\ALpy3109
cwd: c:\PG\AL\ALpy3109
info 13:48:36.385: ipykernel version & path 6.22.0, ~\anaconda3\envs\ALpy3109_1\lib\site-packages\ipykernel\__init__.py for c:\Users\PG\anaconda3\envs\ALpy3109_1\python.exe
info 13:48:37.579: Got new session 308a959c-7d41-4733-b148-0fc441b04181
info 13:48:37.579: Started new restart session
info 13:48:37.579: Dispose Kernel process 22104.
info 13:48:53.419: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:48:53.603: Generated code for 1 = <ipython-input-1-14a4618f255f> with 30 lines
info 13:49:06.962: Generated code for 2 = <ipython-input-2-fa2cd79fb24e> with 1 lines
info 13:49:08.482: Generated code for 3 = <ipython-input-3-507001a65ffe> with 1 lines
info 13:49:17.841: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:49:17.945: Generated code for 4 = <ipython-input-4-041e1b82d363> with 68 lines
info 13:49:20.433: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:49:20.539: Generated code for 5 = <ipython-input-5-9a2fe302d69c> with 10 lines
info 13:49:25.009: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:49:25.095: Generated code for 6 = <ipython-input-6-6f3f27b47c30> with 15 lines
info 13:49:29.218: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:49:29.332: Generated code for 7 = <ipython-input-7-ff77436db5b5> with 6 lines
info 13:49:34.893: Generated code for 8 = <ipython-input-8-a552d8edcebd> with 1 lines
info 13:49:41.457: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:49:41.562: Generated code for 9 = <ipython-input-9-db6dac91fd39> with 41 lines
info 13:49:57.091: Process Execution: > ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
> ~\anaconda3\envs\ALpy3109_1\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\pythonFiles\normalizeSelection.py
info 13:49:57.232: Generated code for 10 = <ipython-input-10-8802c3de338b> with 9 lines
info 13:50:00.233: Generated code for 11 = <ipython-input-11-a7d239cf8e60> with 1 lines
error 13:50:00.450: Disposing session as kernel process died ExitCode: 3221226505, Reason: c:\Users\PG\anaconda3\envs\ALpy3109_1\lib\site-packages\traitlets\traitlets.py:2548: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn(
c:\Users\PG\anaconda3\envs\ALpy3109_1\lib\site-packages\traitlets\traitlets.py:2499: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use '3ff8d1b6-178e-402f-b831-0d24a86dd3cf' instead of 'b"3ff8d1b6-178e-402f-b831-0d24a86dd3cf"'.
  warn(

info 13:50:00.450: Dispose Kernel process 14748.
error 13:50:00.451: Raw kernel process exited code: 3221226505
error 13:50:00.452: Error in waiting for cell to complete [Error: Canceled future for execute_request message before replies were done
	at t.KernelShellFutureHandler.dispose (c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:2:32419)
	at c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:2:51471
	at Map.forEach (<anonymous>)
	at y._clearKernelState (c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:2:51456)
	at y.dispose (c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:2:44938)
	at c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:17:96826
	at ee (c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:2:1589492)
	at jh.dispose (c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:17:96802)
	at Lh.dispose (c:\Users\PG\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:17:104079)
	at process.processTicksAndRejections (node:internal/process/task_queues:96:5)]
warn 13:50:00.453: Cell completed with errors {
  message: 'Canceled future for execute_request message before replies were done'
}
info 13:50:00.454: Cancel all remaining cells true || Error || undefined

In the log it's showing Python 3.9.7 which is the base python.
May I have some help on it please.
Thanks

Prem George · Answer 1 · Mon Apr 17 2023 14:44:40 GMT+0800 (China Standard Time)

I tried this
conda install ipykernel --update-deps --force-reinstall
This updates the following

The following packages will be UPDATED:

  python                                  3.10.9-h966fe2a_2 --> 3.10.10-h966fe2a_2

python interactive update to 3.10.10 before it was 3.10.9

But did not help

Xavier Dupré · Answer 2 · Mon Apr 17 2023 21:00:19 GMT+0800 (China Standard Time)

onnxmltools uses the converter from catboost. Could you check it works without onnxmltools? (see https://catboost.ai/en/docs/concepts/apply-onnx-ml#examples) It it works, this issue come from onnxmltools, otherwise it probably comes from catboost.

Prem George · Answer 3 · Tue Apr 18 2023 07:41:41 GMT+0800 (China Standard Time)

Thanks @xadupre for the quick reply, much appreciated

Yes catboost convertor worked without onnxmltools

Prem George · Answer 4 · Thu Apr 20 2023 10:27:40 GMT+0800 (China Standard Time)

Just to add
Same model works in C#

with following packages

Prem George · Answer 5 · Fri May 05 2023 08:24:23 GMT+0800 (China Standard Time)

Currently after saving the catboost onnx model, we can only use the InferenceSession class to load, but to save the model to the database, I need as a model variable instead of Instance of Class, I am not sure how to load the saved onnx model, this is where convert_catboost helps
onnx_model = convert_catboost(catboostmodel, initial_types=initial_type)

Is there a way for that?

Xavier Dupré · Answer 6 · Fri May 05 2023 14:26:26 GMT+0800 (China Standard Time)

To make sure i understand, you would like to avoid saving the onnx file in disk?

Prem George · Answer 7 · Fri May 05 2023 15:04:23 GMT+0800 (China Standard Time)

No @xadupre catboost model.save_model saving the model to the disk.

After that I want load back the model.onnx file and convert to binary format to save in the database,

But I am not sure how to load back the model.onnx,

the InferenceSession is just loading the class instance, not the model object.

for example:
for xgboost, following onnx_model is the python object, which I can convert to binary file and save in database
onnx_model = convert_xgboost(model, initial_types=initial_type)

But for catboost, I can do only InferenceSession like below, because convert_catboost crashing
sess = onnxruntime.InferenceSession("catboostmodel_test.onnx")
this session is not the python object, just the instance which I can't convert to binary.

Many Thanks

Prem George · Answer 8 · Fri May 12 2023 08:55:30 GMT+0800 (China Standard Time)

Hi @xadupre May I know whether the requirement is correct? Is there any other way to load the onnx file as python object?

Xavier Dupré · Answer 9 · Fri May 12 2023 23:28:36 GMT+0800 (China Standard Time)

import onnx
with open(<filename>, "rb") as f:
  onx = onnx.load(f)

onx is a python object containing the onnx graph. You will find more in this tutorial: https://onnx.ai/onnx/intro/python.html.

Prem George · Answer 10 · Thu Dec 07 2023 13:58:29 GMT+0800 (China Standard Time)

Hi, Can I have some help on this please, Thanks
Still that issue exists,
using

python 3.10.11
catboost 1.1.1
onnxmltools 1.11.2

here is the simple example,

from catboost import CatBoostClassifier
from onnxmltools.convert.common.data_types import FloatTensorType
from onnxmltools.convert import convert_catboost

train_data = [[0, 3], [4, 1], [8, 1], [9, 1]]
train_labels = [0, 0, 1, 1]

model = CatBoostClassifier()
model.fit(train_data,train_labels,verbose=False)

initial_type = [('float_input', FloatTensorType([1, 2]))]
onnx_model = convert_catboost(model, initial_types=initial_type)

if we run the code we get

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click [here](https://aka.ms/vscodeJupyterKernelCrash) for more info. View Jupyter [log](command:jupyter.viewOutput) for further details.

Xavier Dupré · Answer 11 · Thu Dec 07 2023 18:35:11 GMT+0800 (China Standard Time)

catboost updated its code and the last version of the exporter is written in C++. It is no longer possible to use a non public function from catboost to avoid saving the model on disk. I suggest using what catboost recommends. I tried this and it works.

import numpy as np
import onnx
from catboost import CatBoostClassifier
from onnxmltools.convert.common.data_types import FloatTensorType
from onnxmltools.convert import convert_catboost

train_data = [[0, 3], [4, 1], [8, 1], [9, 1]]
train_labels = [0, 0, 1, 1]
X = np.array(train_data, dtype=np.float32)

model = CatBoostClassifier()
model.fit(train_data, train_labels, verbose=False)
expected = model.predict(X)
print("EXPECTED", expected.shape, expected.dtype, expected)


initial_type = [("float_input", FloatTensorType([1, 2]))]
print("conversion")
# fails
# onnx_model = convert_catboost(model, initial_types=initial_type)
# catboost updated its code and the last version of the exporter is
# written in C++. It is no longer possible to use a non public function
# to avoid saving the model on disk.

# This instruction must be used instead.
model.save_model("model.onnx", format="onnx")
print("done")


from onnxruntime import InferenceSession

sess = InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
results = sess.run(None, {"features": np.array(train_data, dtype=np.float32)})[0]
print("ORT", results.shape, results.dtype, results)

Prem George · Answer 12 · Fri Dec 08 2023 07:06:29 GMT+0800 (China Standard Time)

Thanks for checking that and suggestions. Much appreciated.