dgl._ffi.base.DGLError: Index data must be an int64 vector
gerrardmai opened this issue · comments
some problem with the code, is that the version problem? Since i cannot install 1.4.0 verion of torch. only can install version 1.7.1. other pacakges version is the same.
(base) G:\gibh\SINGLE CELL\jiazi\predict_celltype\scDeepSort-updated_pretrained_models>python train.py --species human --tissue Adipose --gpu -1 --filetype gz
Using backend: pytorch
{'batch_size': 500,
'dense_dim': 400,
'dropout': 0.1,
'exclude_rate': 0.005,
'filetype': 'gz',
'gpu': -1,
'hidden_dim': 200,
'lr': 0.001,
'n_epochs': 300,
'n_layers': 1,
'num_neighbors': 0,
'random_seed': 10086,
'species': 'human',
'test_rate': 0.2,
'threshold': 0,
'tissue': 'Adipose',
'unsure_rate': 2.0,
'weight_decay': 0.0005}
The build graph contains 16397 genes with 4 labels supported.
human_Adipose3_data.gz -> Nonzero Ratio: 2.75%
Added 1354 nodes and 610321 edges.
#Nodes in Graph: 17751, #Edges: 1220642.
[PCA] Gene EVR: 66.25 %.
------Train label statistics------
#1 [Stromal cell]: 184
#2 [Fibroblast]: 53
#3 [Macrophage]: 1099
#4 [Monocyte]: 18
Train Number: 1084, Test Number: 270
Traceback (most recent call last):
File "train.py", line 169, in
trainer.fit()
File "train.py", line 47, in fit
loss = self.train()
File "train.py", line 71, in train
for batch, nf in enumerate(NeighborSampler(g=self.graph,
File "D:\anaconda\lib\site-packages\dgl\contrib\sampling\sampler.py", line 314, in init
super(NeighborSampler, self).init(
File "D:\anaconda\lib\site-packages\dgl\contrib\sampling\sampler.py", line 166, in init
self._seed_nodes = utils.toindex(self._seed_nodes)
File "D:\anaconda\lib\site-packages\dgl\utils.py", line 242, in toindex
return data if isinstance(data, Index) else Index(data)
File "D:\anaconda\lib\site-packages\dgl\utils.py", line 15, in init
self._initialize_data(data)
File "D:\anaconda\lib\site-packages\dgl\utils.py", line 22, in _initialize_data
self._dispatch(data)
File "D:\anaconda\lib\site-packages\dgl\utils.py", line 47, in _dispatch
raise DGLError('Index data must be an int64 vector, but got: %s' % str(data))
dgl._ffi.base.DGLError: Index data must be an int64 vector, but got: tensor([17413, 17532, 17478, ..., 16965, 17085, 16614], dtype=torch.int32)
Hi,
I also had troubles running the code in the beginning. Only after I made sure that I had the exact required versions installed, it worked. Try to setup your conda environment like this:
conda create -n scDeepSort python=3.6 scipy==1.3.1 numpy==1.17.2 pandas==0.25.1 scikit-learn==0.22.2.post1 xlrd==1.2.0
conda install pytorch==1.4.0 torchvision==0.5.0 cpuonly -c pytorch
conda install -c dglteam "dgl<0.5"
Cheers,
Mario
Hi,
I also had troubles running the code in the beginning. Only after I made sure that I had the exact required versions installed, it worked. Try to setup your conda environment like this:conda create -n scDeepSort python=3.6 scipy==1.3.1 numpy==1.17.2 pandas==0.25.1 scikit-learn==0.22.2.post1 xlrd==1.2.0 conda install pytorch==1.4.0 torchvision==0.5.0 cpuonly -c pytorch conda install -c dglteam "dgl<0.5"Cheers,
Mario
HI,
I follow your code to creat a new env, but the same error occur.
The new env is shown below:
(scDeepSort) G:\gibh\SINGLE CELL\jiazi\predict_celltype\scDeepSort-updated_pretrained_models>conda list
blas 1.0 mkl
certifi 2020.12.5 py36haa95532_0
cffi 1.14.4 py36hcd4344a_0
cpuonly 1.0 0 pytorch
decorator 4.4.2 py_0
dgl 0.4.3post2 py36_0 dglteam
freetype 2.10.4 hd328e21_0
icc_rt 2019.0.0 h0cc432a_1
intel-openmp 2020.2 254
joblib 0.14.1 py_0 conda-forge/label/cf202003
jpeg 9b hb83a4c4_2
libblas 3.8.0 7_h8933c1f_netlib conda-forge/label/cf202003
libcblas 3.8.0 7_h8933c1f_netlib conda-forge/label/cf202003
libpng 1.6.37 h2a8f88b_0
libtiff 4.1.0 h56a325e_1
lz4-c 1.9.2 hf4a77e7_3
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
mkl 2020.2 256
mkl-service 2.3.0 py36h196d8e1_0
mkl_fft 1.2.0 py36h45dec08_0
mkl_random 1.1.1 py36h47e9c7a_0
msys2-conda-epoch 20160418 1
networkx 2.5 py_0
ninja 1.10.2 py36h6d14046_0
numpy 1.17.2 py36hc73f44b_0
numpy-base 1.17.2 py36hc3f5095_0
olefile 0.46 py36_0
pandas 0.25.1 py36ha925a31_0
pillow 8.0.1 py36h4fa10fc_0
pip 20.3.3 py36haa95532_0
pycparser 2.20 py_2
python 3.6.12 h5500b2f_2
python-dateutil 2.8.1 py_0
python_abi 3.6 1_cp36m conda-forge/label/cf202003
pytorch 1.4.0 py3.6_cpu_0 [cpuonly] pytorch
pytz 2020.4 pyhd3eb1b0_0
scikit-learn 0.22.2.post1 py36h7208079_0 conda-forge/label/cf202003
scipy 1.3.1 py36h29ff71c_0
setuptools 51.0.0 py36haa95532_2
six 1.15.0 py36haa95532_0
sqlite 3.33.0 h2a8f88b_0
tk 8.6.10 he774522_0
torchvision 0.5.0 py36_cpu [cpuonly] pytorch
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
wheel 0.36.2 pyhd3eb1b0_0
wincertstore 0.2 py36h7fe50ca_0
xlrd 1.2.0 py36_0
xz 5.2.5 h62dcd97_0
zlib 1.2.11 h62dcd97_4
zstd 1.4.5 h04227a9_0
is that other required pacakge should be installed? could you provied your env list to me? I try to creat a new env which is same of yours.
HI,
I follow your code to creat a new env, but the same error occur.
The new env is shown below:
(scDeepSort) G:\gibh\SINGLE CELL\jiazi\predict_celltype\scDeepSort-updated_pretrained_models>conda list
blas 1.0 mkl
certifi 2020.12.5 py36haa95532_0
cffi 1.14.4 py36hcd4344a_0
cpuonly 1.0 0 pytorch
decorator 4.4.2 py_0
dgl 0.4.3post2 py36_0 dglteam
freetype 2.10.4 hd328e21_0
icc_rt 2019.0.0 h0cc432a_1
intel-openmp 2020.2 254
joblib 0.14.1 py_0 conda-forge/label/cf202003
jpeg 9b hb83a4c4_2
libblas 3.8.0 7_h8933c1f_netlib conda-forge/label/cf202003
libcblas 3.8.0 7_h8933c1f_netlib conda-forge/label/cf202003
libpng 1.6.37 h2a8f88b_0
libtiff 4.1.0 h56a325e_1
lz4-c 1.9.2 hf4a77e7_3
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
mkl 2020.2 256
mkl-service 2.3.0 py36h196d8e1_0
mkl_fft 1.2.0 py36h45dec08_0
mkl_random 1.1.1 py36h47e9c7a_0
msys2-conda-epoch 20160418 1
networkx 2.5 py_0
ninja 1.10.2 py36h6d14046_0
numpy 1.17.2 py36hc73f44b_0
numpy-base 1.17.2 py36hc3f5095_0
olefile 0.46 py36_0
pandas 0.25.1 py36ha925a31_0
pillow 8.0.1 py36h4fa10fc_0
pip 20.3.3 py36haa95532_0
pycparser 2.20 py_2
python 3.6.12 h5500b2f_2
python-dateutil 2.8.1 py_0
python_abi 3.6 1_cp36m conda-forge/label/cf202003
pytorch 1.4.0 py3.6_cpu_0 [cpuonly] pytorch
pytz 2020.4 pyhd3eb1b0_0
scikit-learn 0.22.2.post1 py36h7208079_0 conda-forge/label/cf202003
scipy 1.3.1 py36h29ff71c_0
setuptools 51.0.0 py36haa95532_2
six 1.15.0 py36haa95532_0
sqlite 3.33.0 h2a8f88b_0
tk 8.6.10 he774522_0
torchvision 0.5.0 py36_cpu [cpuonly] pytorch
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
wheel 0.36.2 pyhd3eb1b0_0
wincertstore 0.2 py36h7fe50ca_0
xlrd 1.2.0 py36_0
xz 5.2.5 h62dcd97_0
zlib 1.2.11 h62dcd97_4
zstd 1.4.5 h04227a9_0
is that other required pacakge should be installed? could you provied your env list to me? I try to creat a new env which is same of yours.
Hi,
here is the output of conda list:
|Name|Version|Build|Channel
_libgcc_mutex|0.1|conda_forge|conda-forge
_openmp_mutex|4.5|1_gnu|conda-forge
ca-certificates|2020.12.5|ha878542_0|conda-forge
certifi|2020.12.5|py36h5fab9bb_0|conda-forge
cpuonly|1.0|0|pytorch
decorator|4.4.2|py_0|conda-forge
dgl|0.4.3post2|py36_0|dglteam
freetype|2.10.4|h7ca028e_0|conda-forge
intel-openmp|2020.2|254|
joblib|0.17.0|py_0|conda-forge
jpeg|9d|h36c2ea0_0|conda-forge
lcms2|2.11|hcbb858e_1|conda-forge
ld_impl_linux-64|2.35.1|hed1e6ac_0|conda-forge
libblas|3.9.0|3_openblas|conda-forge
libcblas|3.9.0|3_openblas|conda-forge
libffi|3.3|h58526e2_2|conda-forge
libgcc-ng|9.3.0|h5dbcf3e_17|conda-forge
libgfortran-ng|7.5.0|hae1eefd_17|conda-forge
libgfortran4|7.5.0|hae1eefd_17|conda-forge
libgomp|9.3.0|h5dbcf3e_17|conda-forge
liblapack|3.9.0|3_openblas|conda-forge
libopenblas|0.3.12|pthreads_hb3c22a3_1|conda-forge
libpng|1.6.37|h21135ba_2|conda-forge
libstdcxx-ng|9.3.0|h2ae2ef3_17|conda-forge
libtiff|4.1.0|h4f3a223_6|conda-forge
libwebp-base|1.1.0|h36c2ea0_3|conda-forge
lz4-c|1.9.2|he1b5a44_3|conda-forge
mkl|2020.2|256|
ncurses|6.2|h58526e2_4|conda-forge
networkx|2.5|py_0|conda-forge
ninja|1.10.2|h4bd325d_0|conda-forge
numpy|1.17.2|py36h95a1406_0|conda-forge
olefile|0.46|pyh9f0ad1d_1|conda-forge
openssl|1.1.1h|h516909a_0|conda-forge
pandas|0.25.1|py36hb3f55d8_0|conda-forge
pillow|8.0.1|py36h10ecd5c_0|conda-forge
pip|20.3.1|pyhd8ed1ab_0|conda-forge
python|3.6.11|hffdb5ce_3_cpython|conda-forge
python-dateutil|2.8.1|py_0|conda-forge
python_abi|3.6|1_cp36m|conda-forge
pytorch|1.4.0|py3.6_cpu_0|[cpuonly]|pytorch
pytz|2020.4|pyhd8ed1ab_0|conda-forge
readline|8.0|he28a2e2_2|conda-forge
scikit-learn|0.22.2.post1|py36hcdab131_0|conda-forge
scipy|1.3.1|py36h921218d_2|conda-forge
setuptools|49.6.0|py36h9880bd3_2|conda-forge
six|1.15.0|pyh9f0ad1d_0|conda-forge
sqlite|3.34.0|h74cdb3f_0|conda-forge
tk|8.6.10|hed695b0_1|conda-forge
torchvision|0.5.0|py36_cpu|[cpuonly]|pytorch
wheel|0.36.1|pyhd3deb0d_0|conda-forge
xlrd|1.2.0|pyh9f0ad1d_1|conda-forge
xz|5.2.5|h516909a_1|conda-forge
zlib|1.2.11|h516909a_1010|conda-forge
zstd|1.4.5|h6597ccf_2|conda-forge
Hi,
here is the output of conda list:
|Name|Version|Build|Channel
_libgcc_mutex|0.1|conda_forge|conda-forge
_openmp_mutex|4.5|1_gnu|conda-forge
ca-certificates|2020.12.5|ha878542_0|conda-forge
certifi|2020.12.5|py36h5fab9bb_0|conda-forge
cpuonly|1.0|0|pytorch
decorator|4.4.2|py_0|conda-forge
dgl|0.4.3post2|py36_0|dglteam
freetype|2.10.4|h7ca028e_0|conda-forge
intel-openmp|2020.2|254|
joblib|0.17.0|py_0|conda-forge
jpeg|9d|h36c2ea0_0|conda-forge
lcms2|2.11|hcbb858e_1|conda-forge
ld_impl_linux-64|2.35.1|hed1e6ac_0|conda-forge
libblas|3.9.0|3_openblas|conda-forge
libcblas|3.9.0|3_openblas|conda-forge
libffi|3.3|h58526e2_2|conda-forge
libgcc-ng|9.3.0|h5dbcf3e_17|conda-forge
libgfortran-ng|7.5.0|hae1eefd_17|conda-forge
libgfortran4|7.5.0|hae1eefd_17|conda-forge
libgomp|9.3.0|h5dbcf3e_17|conda-forge
liblapack|3.9.0|3_openblas|conda-forge
libopenblas|0.3.12|pthreads_hb3c22a3_1|conda-forge
libpng|1.6.37|h21135ba_2|conda-forge
libstdcxx-ng|9.3.0|h2ae2ef3_17|conda-forge
libtiff|4.1.0|h4f3a223_6|conda-forge
libwebp-base|1.1.0|h36c2ea0_3|conda-forge
lz4-c|1.9.2|he1b5a44_3|conda-forge
mkl|2020.2|256|
ncurses|6.2|h58526e2_4|conda-forge
networkx|2.5|py_0|conda-forge
ninja|1.10.2|h4bd325d_0|conda-forge
numpy|1.17.2|py36h95a1406_0|conda-forge
olefile|0.46|pyh9f0ad1d_1|conda-forge
openssl|1.1.1h|h516909a_0|conda-forge
pandas|0.25.1|py36hb3f55d8_0|conda-forge
pillow|8.0.1|py36h10ecd5c_0|conda-forge
pip|20.3.1|pyhd8ed1ab_0|conda-forge
python|3.6.11|hffdb5ce_3_cpython|conda-forge
python-dateutil|2.8.1|py_0|conda-forge
python_abi|3.6|1_cp36m|conda-forge
pytorch|1.4.0|py3.6_cpu_0|[cpuonly]|pytorch
pytz|2020.4|pyhd8ed1ab_0|conda-forge
readline|8.0|he28a2e2_2|conda-forge
scikit-learn|0.22.2.post1|py36hcdab131_0|conda-forge
scipy|1.3.1|py36h921218d_2|conda-forge
setuptools|49.6.0|py36h9880bd3_2|conda-forge
six|1.15.0|pyh9f0ad1d_0|conda-forge
sqlite|3.34.0|h74cdb3f_0|conda-forge
tk|8.6.10|hed695b0_1|conda-forge
torchvision|0.5.0|py36_cpu|[cpuonly]|pytorch
wheel|0.36.1|pyhd3deb0d_0|conda-forge
xlrd|1.2.0|pyh9f0ad1d_1|conda-forge
xz|5.2.5|h516909a_1|conda-forge
zlib|1.2.11|h516909a_1010|conda-forge
zstd|1.4.5|h6597ccf_2|conda-forge
@mariokreutzfeldt
Thank you for your help.
I found that it was not the pacakge version problem.
I change seed_nodes=self.train_ids to self.train.ids.numpy() , it works.
could you provide me your train.py? I want to check it again.
@gerrardmai , no problem.
Cool that it works, now.
I am running the train.py from the repo. Didn't do any edits.
Cheers,
Mario
Hi,
here is the output of conda list:
|Name|Version|Build|Channel
_libgcc_mutex|0.1|conda_forge|conda-forge
_openmp_mutex|4.5|1_gnu|conda-forge
ca-certificates|2020.12.5|ha878542_0|conda-forge
certifi|2020.12.5|py36h5fab9bb_0|conda-forge
cpuonly|1.0|0|pytorch
decorator|4.4.2|py_0|conda-forge
dgl|0.4.3post2|py36_0|dglteam
freetype|2.10.4|h7ca028e_0|conda-forge
intel-openmp|2020.2|254|
joblib|0.17.0|py_0|conda-forge
jpeg|9d|h36c2ea0_0|conda-forge
lcms2|2.11|hcbb858e_1|conda-forge
ld_impl_linux-64|2.35.1|hed1e6ac_0|conda-forge
libblas|3.9.0|3_openblas|conda-forge
libcblas|3.9.0|3_openblas|conda-forge
libffi|3.3|h58526e2_2|conda-forge
libgcc-ng|9.3.0|h5dbcf3e_17|conda-forge
libgfortran-ng|7.5.0|hae1eefd_17|conda-forge
libgfortran4|7.5.0|hae1eefd_17|conda-forge
libgomp|9.3.0|h5dbcf3e_17|conda-forge
liblapack|3.9.0|3_openblas|conda-forge
libopenblas|0.3.12|pthreads_hb3c22a3_1|conda-forge
libpng|1.6.37|h21135ba_2|conda-forge
libstdcxx-ng|9.3.0|h2ae2ef3_17|conda-forge
libtiff|4.1.0|h4f3a223_6|conda-forge
libwebp-base|1.1.0|h36c2ea0_3|conda-forge
lz4-c|1.9.2|he1b5a44_3|conda-forge
mkl|2020.2|256|
ncurses|6.2|h58526e2_4|conda-forge
networkx|2.5|py_0|conda-forge
ninja|1.10.2|h4bd325d_0|conda-forge
numpy|1.17.2|py36h95a1406_0|conda-forge
olefile|0.46|pyh9f0ad1d_1|conda-forge
openssl|1.1.1h|h516909a_0|conda-forge
pandas|0.25.1|py36hb3f55d8_0|conda-forge
pillow|8.0.1|py36h10ecd5c_0|conda-forge
pip|20.3.1|pyhd8ed1ab_0|conda-forge
python|3.6.11|hffdb5ce_3_cpython|conda-forge
python-dateutil|2.8.1|py_0|conda-forge
python_abi|3.6|1_cp36m|conda-forge
pytorch|1.4.0|py3.6_cpu_0|[cpuonly]|pytorch
pytz|2020.4|pyhd8ed1ab_0|conda-forge
readline|8.0|he28a2e2_2|conda-forge
scikit-learn|0.22.2.post1|py36hcdab131_0|conda-forge
scipy|1.3.1|py36h921218d_2|conda-forge
setuptools|49.6.0|py36h9880bd3_2|conda-forge
six|1.15.0|pyh9f0ad1d_0|conda-forge
sqlite|3.34.0|h74cdb3f_0|conda-forge
tk|8.6.10|hed695b0_1|conda-forge
torchvision|0.5.0|py36_cpu|[cpuonly]|pytorch
wheel|0.36.1|pyhd3deb0d_0|conda-forge
xlrd|1.2.0|pyh9f0ad1d_1|conda-forge
xz|5.2.5|h516909a_1|conda-forge
zlib|1.2.11|h516909a_1010|conda-forge
zstd|1.4.5|h6597ccf_2|conda-forge@mariokreutzfeldt Thank you for your help.
I found that it was not the pacakge version problem.
I change seed_nodes=self.train_ids to self.train.ids.numpy() , it works.
could you provide me your train.py? I want to check it again.
using pip install -U dgl-cu102==0.4.3 -f https://data.dgl.ai/wheels/repo.html ,
change seed_nodes=self.train_ids to seed_nodes=train_ids.cpu().numpy()) and valid_ids.cpu().numpy() works for me
Hello, I'm getting the same error:
DGLError: Index data must be an int64 vector, but got: tensor([15089, 16792, 16182, ..., 12570, 15093, 16394], dtype=torch.int32)
I have followed all the steps on this thread, here's my conda list:
Name Version Build Channel
blas 1.0 mkl
certifi 2021.5.30 py36haa95532_0
charset-normalizer 2.0.12 pypi_0 pypi
colorama 0.4.5 pypi_0 pypi
dataclasses 0.8 pypi_0 pypi
decorator 5.1.1 pyhd3eb1b0_0
deepsort 1.0 pypi_0 pypi
dgl 0.4.3 pypi_0 pypi
icc_rt 2022.1.0 h6049295_2
idna 3.7 pypi_0 pypi
importlib-resources 5.4.0 pypi_0 pypi
intel-openmp 2023.1.0 h59b6b97_46320
joblib 1.0.1 pyhd3eb1b0_0
mkl 2020.2 256
mkl-service 2.3.0 py36h196d8e1_0
mkl_fft 1.3.0 py36h46781fe_0
mkl_random 1.1.1 py36h47e9c7a_0
networkx 2.5 py_0
numpy 1.17.2 py36hc73f44b_0
numpy-base 1.17.2 py36hc3f5095_0
pandas 0.25.1 py36ha925a31_0
pip 21.2.2 py36haa95532_0
python 3.6.13 h3758d61_0
python-dateutil 2.8.2 pyhd3eb1b0_0
pytz 2021.3 pyhd3eb1b0_0
requests 2.27.1 pypi_0 pypi
scikit-learn 0.24.2 py36hf11a4ad_1
scipy 1.3.1 py36h29ff71c_0
setuptools 58.0.4 py36haa95532_0
six 1.16.0 pyhd3eb1b0_1
sqlite 3.45.3 h2bbff1b_0
threadpoolctl 2.2.0 pyh0d69192_0
torch 1.7.1 pypi_0 pypi
tqdm 4.64.1 pypi_0 pypi
typing-extensions 4.1.1 pypi_0 pypi
urllib3 1.26.18 pypi_0 pypi
vc 14.2 h2eaa2aa_1
vs2015_runtime 14.29.30133 h43f2093_3
wheel 0.37.1 pyhd3eb1b0_0
wincertstore 0.2 py36h7fe50ca_0
xlrd 1.2.0 py36_0
zipp 3.6.0 pypi_0 pypi
Please let me know if you have found a fix
Thanks