a question about why I can't get any points from the frame.py?

[laliwang]

  I may have successfully run this code's initialization step under the scannet scene0000_00 datasets, while after the initialization step, something wrong happen in the process of mapping as follows:

root@autodl-container-e1cd11a652-b3827dab:/home/Vox-Fusion# python demo/run.py configs/scannet/scene0000.yaml
******* initializing first_frame: 0
initializing the first frame ...
mapping process started!
Process Process-2:
Traceback (most recent call last):
File "/root/miniconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/root/miniconda3/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/Vox-Fusion/src/mapping.py", line 93, in spin
self.create_voxels(tracked_frame)
File "/home/Vox-Fusion/src/mapping.py", line 201, in create_voxels
points = frame.get_points().cuda()
File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/Vox-Fusion/src/frame.py", line 64, in get_points
vmap = self.rays_d * self.depth[..., None]
RuntimeError: The size of tensor a (468) must match the size of tensor b (628) at non-singleton dimension 1
******* tracking process started! *******
tracking frame: 0%| | 0/5577 [00:00<?, ?it/s]error in dataloading: 'NoneType' object has no attribute 'cuda' skipping frame 1
error in dataloading: 'NoneType' object has no attribute 'cuda' skipping frame 2
error in dataloading: 'NoneType' object has no attribute 'cuda' skipping frame 3
error in dataloading: 'NoneType' object has no attribute 'cuda' skipping frame 4
tracking frame: 0%| | 4/5577 [00:00<02:53, 32.17it/s]

  From the above traceback, I may indicate that there's something wrong with the function "get_points()" in frame.py, so that I cannot get proper data for the following mapping process. While I have finished the building of third_party in your folders, and the environment of my computer is python3.8 + pytorch 1.8.1 + cuda11.1, and I am sure that I have put the scannet datasets in the correct path, so I don't know why the code cannot get points properly from the provided depth images. 
   Could you please give me some guidance on how to solve this errors? Thank you very much!

[garylidd]

Hi, It seems that there is something wrong with the file reading, our file structure should be consistent with the official one.

For example:

• scene0000_00
  • color
    • xxx.jpg
  • depth
    • xxx.png
  • pose
    • xxx.txt
  • intrinsic
    • intrinsic_depth.txt

Please check.

[laliwang]

Thanks for your reply sincerely! While I manage my scannet datasets folder as follows, it seems that I have the same structure with yours supported, and under the same scannet datasets I've successfully run another sequential mapping algorithm, so I think there's no wrong with the file reading. Is there any other reasons for this situation?

[Yang Xing Rui]

Hi, it is difficult to tell but I suspect data loading is an issue.
Can you check if the images are correctly loaded by either printing out their data or saving them as jpeg/png files?

[laliwang]

Hi, it is difficult to tell but I suspect data loading is an issue. Can you check if the images are correctly loaded by either printing out their data or saving them as jpeg/png files?

Thanks for your reply and I would try it soon.

[laliwang]

Hi, I have followed your advice to print out their data in format of tensor using the scannet.py script in your src/dataset folder. But I notice that there is no critical error with the specific value in the matrix. In the same method, I have tried the room_0 sequence from Nice-SLAM under your demo, the results still indicate that there's something wrong with the data reading, which makes me confused these days.
While when I first attempt to run your code in the environment of Pytorch1.8.1, the frame.py interrupt me and report that **TypeError: meshgrid() got an unexpected keyword argument ‘indexing‘**, so I just delete the param of indexing='ij' and keep going. I don't know if this makes anything wrong.

[garylidd]

Sorry I didn't notice the environment, our method is built with pytorch version >= 1.10, an old version of pytroch may have some incompatible features.

[Yang Xing Rui]

Hi, I have followed your advice to print out their data in format of tensor using the scannet.py script in your src/dataset folder. But I notice that there is no critical error with the specific value in the matrix. In the same method, I have tried the room_0 sequence from Nice-SLAM under your demo, the results still indicate that there's something wrong with the data reading, which makes me confused these days.
While when I first attempt to run your code in the environment of Pytorch1.8.1, the frame.py interrupt me and report that TypeError: meshgrid() got an unexpected keyword argument ‘indexing‘, so I just delete the param of indexing='ij' and keep going. I don't know if this makes anything wrong.

torch.meshgrid defaults to indexing='ij' so there shouldn't be any problem.
That sounds weird, if data loading is fine, then you should be getting correct 3D points. Can you provide a more concrete example?

[laliwang]

Sorry I didn't notice the environment, our method is built with pytorch version >= 1.10, an old version of pytroch may have some incompatible features.

Truly thanks for your valuable suggestion! The above data reading problem is indeed due to the mismatch of pytorch version.

[renwuli]

I have the same issue but my pytorch is 1.12.0 > 1.10.0