fail to inference and cannot find where to input the driven image
AWangji opened this issue · comments
hi, thanks for your excellent project! But when I try to inference your model with my video, it failed:
I dont know why. And as you says in README:
only input one video? So where is the driven images?
As in README.md
- Place the video for inference under
SMPLer-X/demo/videos - Prepare the pretrained models to be used for inference under
SMPLer-X/pretrained_models - Prepare the mmdet pretrained model and config under
SMPLer-X/pretrained_models - Inference output will be saved in
SMPLer-X/demo/results
Please check these above path, see if problem still exists.
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
As in README.md
- Place the video for inference under
SMPLer-X/demo/videos- Prepare the pretrained models to be used for inference under
SMPLer-X/pretrained_models- Prepare the mmdet pretrained model and config under
SMPLer-X/pretrained_models- Inference output will be saved in
SMPLer-X/demo/resultsPlease check these above path, see if problem still exists.
Hi, thanks for your suggestion. I have resolved all questions above. Now it can inference successfully.
But I still doubt about how to transfer the action in the video to another target image. Could you give some advice?
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Yeah, I have resolved the previous issue, but since I am using a Windows system, I encountered some new issues while running shell commands. Are you using a Linux system, and should I try again on the same system?
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Yeah, I have resolved the previous issue, but since I am using a Windows system, I encountered some new issues while running shell commands. Are you using a Linux system, and should I try again on the same system?
I encountered a problem while running the srun command, it seems that I did not configure my slurm correctly
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Yeah, I have resolved the previous issue, but since I am using a Windows system, I encountered some new issues while running shell commands. Are you using a Linux system, and should I try again on the same system?
I encountered a problem while running the srun command, it seems that I did not configure my slurm correctly
yes, so do I. I directly use python to execute the command instead. It also work.
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Yeah, I have resolved the previous issue, but since I am using a Windows system, I encountered some new issues while running shell commands. Are you using a Linux system, and should I try again on the same system?
I encountered a problem while running the srun command, it seems that I did not configure my slurm correctly
do you have any idea that we can input a image to rebuild the detected body in the inferenced result video？
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Yeah, I have resolved the previous issue, but since I am using a Windows system, I encountered some new issues while running shell commands. Are you using a Linux system, and should I try again on the same system?
hello i meet the same problem, How did you solve it
Hi, i have noticed that usage of "slum_inference" could be troublesome, i will push a more straight forward script for inference.
Before that, you could try the command from my personal pipeline as below:
cmd_smplerx_inference = f'cd smplerx/main && python inference.py ' \
f'--num_gpus 1 --pretrained_model {args.ckpt} ' \
f'--agora_benchmark agora_model ' \
f'--img_path ../../{frame_path} --start {start_count} --end {end_count} ' \
f'--output_folder ../../{args.save_dir}/{vid_name} ' \
f'--show_verts --show_bbox '
- "args.ckpt" is one of the models, e.g.'smpler_x_h32'
- "--img_path" is where you save the split frames with name"%06d.jpg"
- "--start" & "--end" refers to the frame number you wish to process
- "--output_folder" is where the results save
And as for OS, we have been developing on Linux, Ubuntu18.04 & 20.04, for windows users I recommend WSL2 & Ubuntu 20.04, which is my environment and inference pipeline is well tested.
For projection of results, please refer to "2D Smplx Overlay" in "readme.md".
Hi, i have noticed that usage of "slum_inference" could be troublesome, i will push a more straight forward script for inference. Before that, you could try the command from my personal pipeline as below:
cmd_smplerx_inference = f'cd smplerx/main && python inference.py ' \ f'--num_gpus 1 --pretrained_model {args.ckpt} ' \ f'--agora_benchmark agora_model ' \ f'--img_path ../../{frame_path} --start {start_count} --end {end_count} ' \ f'--output_folder ../../{args.save_dir}/{vid_name} ' \ f'--show_verts --show_bbox '
- "args.ckpt" is one of the models, e.g.'smpler_x_h32'
- "--img_path" is where you save the split frames with name"%06d.jpg"
- "--start" & "--end" refers to the frame number you wish to process
- "--output_folder" is where the results save
And as for OS, we have been developing on Linux, Ubuntu18.04 & 20.04, for windows users I recommend WSL2 & Ubuntu 20.04, which is my environment and inference pipeline is well tested.
For projection of results, please refer to "2D Smplx Overlay" in "readme.md".
That is to say, you do not support make one image retargeted by a driving video, like this
Hey, I'm encountering the same problem. I have a directory of images, but I'm not sure what content should be placed here. Based on my reading of the homepage, I only know that I need to provide video files
yeah same question. Have you solved it?
Yeah, I have resolved the previous issue, but since I am using a Windows system, I encountered some new issues while running shell commands. Are you using a Linux system, and should I try again on the same system?
I encountered a problem while running the srun command, it seems that I did not configure my slurm correctly
my python command is: python inference.py
--num_gpus ${GPUS_PER_NODE}
--exp_name output/demo_${JOB_NAME}
--pretrained_model ${CKPT}
--agora_benchmark agora_model
--img_path ${IMG_PATH}
--start 1
--end $end_count
--output_folder ${SAVE_DIR}
--show_verts
--show_bbox
--save_mesh
and delete all about "srun". It makes sense. you can try that
嘿，我遇到了同样的问题。我有一个图像目录，但我不确定应该在这里放置什么内容。根据我对主页的阅读，我只知道我需要提供视频文件
是的，同样的问题。你解决了吗？
是的，我已经解决了上一个问题，但是由于我使用的是 Windows 系统，因此在运行 shell 命令时遇到了一些新问题。您使用的是 Linux 系统，我应该在同一系统上重试吗？
我在运行 srun 命令时遇到了问题，似乎我没有正确配置我的 slurm
我的python命令是： python inference.py --num_gpus ${GPUS_PER_NODE} --exp_name output/demo_${JOB_NAME} --pretrained_model ${CKPT} --agora_benchmark agora_model --img_path ${IMG_PATH} --start 1 --end
$end_count --output_folder $ {SAVE_DIR} --show_verts --show_bbox --save_mesh 并删除所有关于“srun”的内容。这是有道理的。你可以试试
嗨，我注意到使用“slum_inference”可能会很麻烦，我将推送一个更直接的脚本进行推理。在此之前，您可以尝试我的个人管道中的命令，如下所示：
cmd_smplerx_inference = f'cd smplerx/main && python inference.py ' \ f'--num_gpus 1 --pretrained_model {args.ckpt} ' \ f'--agora_benchmark agora_model ' \ f'--img_path ../../{frame_path} --start {start_count} --end {end_count} ' \ f'--output_folder ../../{args.save_dir}/{vid_name} ' \ f'--show_verts --show_bbox '
- “args.ckpt”是其中一个模型，例如“smpler_x_h32”
- “--img_path”是保存名称为“%06d.jpg”的拆分帧的位置
- “--start”和“--end”是指您要处理的帧号
- “--output_folder”是结果保存的位置
至于操作系统，我们一直在Linux、Ubuntu18.04和20.04上开发，对于Windows用户，我推荐WSL2和Ubuntu 20.04，这是我的环境，推理管道经过了很好的测试。
有关结果的投影，请参阅“readme.md”中的“2D Smplx Overlay”。
Thank you for your help. It can work now. I have successfully run the inference and I am trying to execute '2D Smplx Overlay', but it seems that my input has not been processed by render.py. Has anyone successfully run this step?
Thanks for this great project, we are trying to get this working in google collab and running into similar problem with path to the videos. Tried your personal pipeline but still not working.
Hi @99bits , seems your problem is with mmcv, please use "mmcv-full" and version stated in the repo.
Besides, I suggest you to search this error message under mmcv repo, seems plenty of people are struggling with mmcv in collab.
As for @HeptagramV , please check if thes image directory exists:
{SMPLERX INFERENCE DIR}/{VIDEO NAME (no extension)}/orig_img/%06d.jpg
And if all the output from inference are placed under:
{SMPLERX INFERENCE DIR}/{VIDEO NAME (no extension)}
Please refer to commands in README.md, the relateive path should be strictly followed.
ffmpeg -i {VIDEO_FILE} -f image2 -vf fps=30 \
{SMPLERX INFERENCE DIR}/{VIDEO NAME (no extension)}/orig_img/%06d.jpg \
-hide_banner -loglevel error
cd main && python render.py \
--data_path {SMPLERX INFERENCE DIR} --seq {VIDEO NAME} \
--image_path {SMPLERX INFERENCE DIR}/{VIDEO NAME} \
--render_biggest_person False
I've been trying to run this pipeline on google colab. So far tried:
- using the exact version of mmcv-full as repo
- running the inference script which got me the same error as this post (check photo attachment)
- tried running the inference.py which gave me "No module named 'mmcv._ext'" error
- looked into the mmcv-full repo with this error message. Suggested solutions didn't work in colab.
- Here is the link to the notepad: https://colab.research.google.com/drive/1nRcZOvKSJAUqictnILWtC1y23_xJ8uX_?usp=sharing
@Wei-Chen-hub can you please help me with this or suggest me a possible solution?
hi, thanks for your excellent project! But when I try to inference your model with my video, it failed:
I dont know why. And as you says in README:
only input one video? So where is the driven images?
Hey, I'm encountering the same problem.it failed:
I have checked README.md but it doesn`t work. I noticed that you have solved the problem, can you help me with this?
Hi, i have noticed that usage of "slum_inference" could be troublesome, i will push a more straight forward script for inference. Before that, you could try the command from my personal pipeline as below:
cmd_smplerx_inference = f'cd smplerx/main && python inference.py ' \ f'--num_gpus 1 --pretrained_model {args.ckpt} ' \ f'--agora_benchmark agora_model ' \ f'--img_path ../../{frame_path} --start {start_count} --end {end_count} ' \ f'--output_folder ../../{args.save_dir}/{vid_name} ' \ f'--show_verts --show_bbox '
- "args.ckpt" is one of the models, e.g.'smpler_x_h32'
- "--img_path" is where you save the split frames with name"%06d.jpg"
- "--start" & "--end" refers to the frame number you wish to process
- "--output_folder" is where the results save
And as for OS, we have been developing on Linux, Ubuntu18.04 & 20.04, for windows users I recommend WSL2 & Ubuntu 20.04, which is my environment and inference pipeline is well tested.
For projection of results, please refer to "2D Smplx Overlay" in "readme.md".
I tried to run the code in ubuntu 20.04 but I got an error with human model. Could anyone help me with this? Thank you in advance!
Command:
python inference.py --num_gpus 1 --exp_name output/demo --pretrained_model smpler_x_h32 --agora_benchmark agora_model --img_path "/home/cc/git/SMPLer-X/demo/images/video2" --start 1 --end 10 --output_folder "/home/cc/git/SMPLer-X" --show_verts --show_bbox
The error message is:
Traceback (most recent call last):
File "inference.py", line 188, in <module>
main()
File "inference.py", line 55, in main
from base import Demoer
File "/home/cc/git/SMPLer-X/main/../common/base.py", line 11, in <module>
from SMPLer_X import get_model
File "/home/cc/git/SMPLer-X/main/../main/SMPLer_X.py", line 4, in <module>
from nets.smpler_x import PositionNet, HandRotationNet, FaceRegressor, BoxNet, HandRoI, BodyRotationNet
File "/home/cc/git/SMPLer-X/main/../common/nets/smpler_x.py", line 6, in <module>
from utils.human_models import smpl_x
File "/home/cc/git/SMPLer-X/main/../common/utils/human_models.py", line 175, in <module>
smpl_x = SMPLX()
File "/home/cc/git/SMPLer-X/main/../common/utils/human_models.py", line 11, in __init__
self.layer = {'neutral': smplx.create(cfg.human_model_path, 'smplx', gender='NEUTRAL', use_pca=False, use_face_contour=True, **self.layer_arg),
File "/home/cc/git/SMPLer-X/main/../common/utils/smplx/smplx/body_models.py", line 2331, in create
raise ValueError(f'Unknown model type {model_type}, exiting!')
ValueError: Unknown model type human, exiting!
Hi, i have noticed that usage of "slum_inference" could be troublesome, i will push a more straight forward script for inference. Before that, you could try the command from my personal pipeline as below:
cmd_smplerx_inference = f'cd smplerx/main && python inference.py ' \ f'--num_gpus 1 --pretrained_model {args.ckpt} ' \ f'--agora_benchmark agora_model ' \ f'--img_path ../../{frame_path} --start {start_count} --end {end_count} ' \ f'--output_folder ../../{args.save_dir}/{vid_name} ' \ f'--show_verts --show_bbox '
- "args.ckpt" is one of the models, e.g.'smpler_x_h32'
- "--img_path" is where you save the split frames with name"%06d.jpg"
- "--start" & "--end" refers to the frame number you wish to process
- "--output_folder" is where the results save
And as for OS, we have been developing on Linux, Ubuntu18.04 & 20.04, for windows users I recommend WSL2 & Ubuntu 20.04, which is my environment and inference pipeline is well tested.
For projection of results, please refer to "2D Smplx Overlay" in "readme.md".I tried to run the code in ubuntu 20.04 but I got an error with human model. Could anyone help me with this? Thank you in advance!
Command:
python inference.py --num_gpus 1 --exp_name output/demo --pretrained_model smpler_x_h32 --agora_benchmark agora_model --img_path "/home/cc/git/SMPLer-X/demo/images/video2" --start 1 --end 10 --output_folder "/home/cc/git/SMPLer-X" --show_verts --show_bboxThe error message is:
Traceback (most recent call last): File "inference.py", line 188, in <module> main() File "inference.py", line 55, in main from base import Demoer File "/home/cc/git/SMPLer-X/main/../common/base.py", line 11, in <module> from SMPLer_X import get_model File "/home/cc/git/SMPLer-X/main/../main/SMPLer_X.py", line 4, in <module> from nets.smpler_x import PositionNet, HandRotationNet, FaceRegressor, BoxNet, HandRoI, BodyRotationNet File "/home/cc/git/SMPLer-X/main/../common/nets/smpler_x.py", line 6, in <module> from utils.human_models import smpl_x File "/home/cc/git/SMPLer-X/main/../common/utils/human_models.py", line 175, in <module> smpl_x = SMPLX() File "/home/cc/git/SMPLer-X/main/../common/utils/human_models.py", line 11, in __init__ self.layer = {'neutral': smplx.create(cfg.human_model_path, 'smplx', gender='NEUTRAL', use_pca=False, use_face_contour=True, **self.layer_arg), File "/home/cc/git/SMPLer-X/main/../common/utils/smplx/smplx/body_models.py", line 2331, in create raise ValueError(f'Unknown model type {model_type}, exiting!') ValueError: Unknown model type human, exiting!
Hi @g-ch ,
please check your file structure, especially this part:
if you don't set it correctly, this piece of code will try to guess your model type, then dir "human_model_files" will be evaluated as "human", boom! no such model type!
btw, all pretrained files and models could be here, I suggest you download them all and place them exactly like what is written in README.md.
hi, thanks for your excellent project! But when I try to inference your model with my video, it failed:
Hey, I'm encountering the same problem.it failed:
I have checked README.md but it doesn`t work. I noticed that you have solved the problem, can you help me with this?
Hi @littleemo ,
I have the same issue, but this could be ignored. Because even shell script failed, ffmpeg already generated a sequence of images for future inferernce. so you can start directly using python script from this step now, you can see how the python command works in this issue :)
if you are working with vscode, here is a launch.json that may help you https://github.com/zacida/SMPLer-X/blob/main/.vscode/launch.json