Segmentation Fault
Hitesh-Nagothu opened this issue · comments
Getting segmentation fault while trying to run the 'python demo_imitator.py --gpu_ids 1'.
Config: Pytorch 1.2/ CUDA 10.0
Providing below the trace on: python demo_imitator.py --gpu_ids 0
Config on HPC:
Device Name Tesla V100-PCIE-16GB
Get Device Count 2
Current Device 0
Is CUDA available(torch.cuda.is_available()) :True
-----------------------------------Trace------------------------------------
------------ Options -------------
T_pose: False
batch_size: 4
bg_ks: 13
bg_model: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth
bg_replace: False
body_seg: False
cam_strategy: smooth
checkpoints_dir: ./outputs/checkpoints/
cond_nc: 3
data_dir: /p300/datasets/iPER
dataset_mode: iPER
debug: False
do_saturate_mask: False
face_model: assets/pretrains/sphere20a_20171020.pth
front_warp: False
ft_ks: 3
gen_name: impersonator
gpu_ids: 0
has_detector: False
hmr_model: assets/pretrains/hmr_tf2pt.pth
image_size: 256
images_folder: images_HD
ip:
is_train: False
load_epoch: 0
load_path: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth
map_name: uv_seg
model: impersonator
n_threads_test: 2
name: running
norm_type: instance
only_vis: False
output_dir: ./outputs/results/
part_info: assets/pretrains/smpl_part_info.json
port: 31100
post_tune: False
pri_path: ./assets/samples/A_priors/imgs
repeat_num: 6
save_res: False
serial_batches: False
smpl_model: assets/pretrains/smpl_model.pkl
smpls_folder: smpls
src_path:
swap_part: body
test_ids_file: val.txt
tex_size: 3
tgt_path:
time_step: 10
train_ids_file: train.txt
uv_mapping: assets/pretrains/mapper.txt
view_params: R=0,90,0/t=0,0,0
-------------- End ----------------
./outputs/checkpoints/running
0%| | 0/3 [00:00<?, ?it/s]Network impersonator was created
Loading net: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth
Network deepfillv2 was created
Loading net: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth
Personalization: meta imitation...
Segmentation fault
have you ever install neural-renderer successfully? Segmentation fault always happens by this reason
Yes I have installed neural-renderer successfully
Please reference the below trace, I was able to successfully install and import neural_renderer. (Running demo_imitator.py)
_Config on HPC:
Device Name Tesla V100-PCIE-16GB
Get Device Count 2
Current Device 0
Is CUDA available(torch.cuda.is_available()) :True_
(lwgan) [hiteshna@xyz impersonator-master]$ python -c "import neural_renderer; print(neural_renderer.version); "
1.1.3
(lwgan) [hiteshna@xyz impersonator-master]$ python demo_imitator.py --gpu_ids 0
------------ Options -------------
T_pose: False
batch_size: 4
bg_ks: 13
bg_model: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth
bg_replace: False
body_seg: False
cam_strategy: smooth
checkpoints_dir: ./outputs/checkpoints/
cond_nc: 3
data_dir: /p300/datasets/iPER
dataset_mode: iPER
debug: False
do_saturate_mask: False
face_model: assets/pretrains/sphere20a_20171020.pth
front_warp: False
ft_ks: 3
gen_name: impersonator
gpu_ids: 0
has_detector: False
hmr_model: assets/pretrains/hmr_tf2pt.pth
image_size: 256
images_folder: images_HD
ip:
is_train: False
load_epoch: 0
load_path: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth
map_name: uv_seg
model: impersonator
n_threads_test: 2
name: running
norm_type: instance
only_vis: False
output_dir: ./outputs/results/
part_info: assets/pretrains/smpl_part_info.json
port: 31100
post_tune: False
pri_path: ./assets/samples/A_priors/imgs
repeat_num: 6
save_res: False
serial_batches: False
smpl_model: assets/pretrains/smpl_model.pkl
smpls_folder: smpls
src_path:
swap_part: body
test_ids_file: val.txt
tex_size: 3
tgt_path:
time_step: 10
train_ids_file: train.txt
uv_mapping: assets/pretrains/mapper.txt
view_params: R=0,90,0/t=0,0,0
-------------- End ----------------
./outputs/checkpoints/running
0%| | 0/3 [00:00<?, ?it/s]Network impersonator was created
Loading net: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth
Network deepfillv2 was created
Loading net: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth
Personalization: meta imitation...
Segmentation fault
Maybe it is caused by different cuda version, (e.g. neural_render is complied by cuda9.0 but pytorch is complied by cuda10.0),
please checkout your local cuda version(cat /usr/local/cuda/version.txt or nvcc --version) and pytorch version(pip show torch)
Checked the CUDA version active and the torch cuda. They indeed are both the same and on 10.0 CUDA. Yet, segfault. Do you require any specifics about the environment config to locate the issue? Thanks for responding.
(lwgan) [hiteshna@cpn impersonator-master]$ python -c "import torch; print(torch.version.cuda)"
10.0.130
(lwgan) [hiteshna@cpn impersonator-master]$ module load cuda/10.0
CUDA 10.0 with cuDNN 7.6.2 has been loaded.
Resolved
@Hitesh-Nagothu Can you please describe how you fixed it? I am getting the same error. Thanks in advance.
@shujonnaha I first checked if the neural_renderer is properly installed. If yes, then checked for the torch Cuda compatibility based on the respective Cuda version. And it might be silly to mention, it was resolved when during the initial execution I did not explicitly assign a gpu_id (as in demo code). I was using a remote HPC and not a root user.
Thanks! Can you please explain a bit more about how you checked if neural_renderer is properly installed? Also, when you say respective cuda version, do you refer to the cuda version used to compile neural_renderer?