Unsupervised training on unlabeled images?

[Haokun Yang]

Hi, my questions are:

1. Does PE mean ground embedding? Is it a strong requisition for training?
2. how can I alter the pipeline to train on my own data with no gt?

My data: video sequences with no depth annotations, lidar measurements, etc.
In the paper I interpreted the conclusion as GEDepth can be utilized without strong supervision. Only learning the ground slope of Adaptive needs GT, whereas Vanilla does not, so these are what I have tried:

1. add new data classes referring to depthformer_v_ddad.py,
2. disabling USE_DYNAMIC_PE, removing 'depth_gt', 'pk_k_gt' from Collect,
3. other minor changes to bypass loading 'depth_gt' and writing it into dict

I got stuck here:

2024-01-23 10:44:53,497 - depth - INFO - workflow: [('train', 1)], max: 38400 iters                           
Traceback (most recent call last):                                                                            
  File "tools/train.py", line 188, in <module>                                                                
    main()                                                                                                    
  File "tools/train.py", line 184, in main                                                                    
    meta=meta)                                                                                                
  File "/home/user/gedepth/depth/apis/train.py", line 121, in train_depther                                
    runner.run(data_loaders, cfg.workflow)                                                                    
  File "/home/user/.conda/envs/GE/lib/python3.7/site-packages/mmcv/runner/iter_based_runner.py", line 133, 
in run                                                                                                        
    iter_runner(iter_loaders[i], **kwargs)                                                                    
  File "/home/user/.conda/envs/GE/lib/python3.7/site-packages/mmcv/runner/iter_based_runner.py", line 60, i
n train                                                                                                       
    outputs = self.model.train_step(data_batch, self.optimizer, **kwargs)                                     
  File "/home/user/.conda/envs/GE/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 67, in 
train_step                                                                                                    
    return self.module.train_step(*inputs[0], **kwargs[0])                                                    
  File "/home/user/gedepth/depth/models/depther/base.py", line 139, in train_step                         
    losses = self(**data_batch)                                                                               
  File "/home/user/.conda/envs/GE/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _ca
ll_impl                                                                                                       
    result = self.forward(*input, **kwargs)                                                                   
  File "/home/user/.conda/envs/GE/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_f
unc                                                                                                           
    return old_func(*args, **kwargs)                                                                          
  File "/home/user/gedepth/depth/models/depther/base.py", line 109, in forward                             
    return self.forward_train(img, img_metas, **kwargs)                                                       
TypeError: forward_train() missing 1 required positional argument: 'depth_gt' 

Thanks a lot!