asyml / vision-transformer-pytorch

Pytorch version of Vision Transformer (ViT) with pretrained models. This is part of CASL (https://casl-project.github.io/) and ASYML project.

Home Page:https://asyml.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

您好,请问怎么把这个用于视频分类?比如5分钟的短视频

dotsonliu opened this issue · comments

commented
您好,请问怎么把这个用于视频分类?比如5分钟的短视频

Thanks for your question. We haven't applied the model to video classification. However, you could use the ViT as a base model to encode each frame of your video.

commented

when there has hundreds of frames ,How to deal with it?

Thank you for your question. The problem of how to efficiently process videos is interesting, but not the focus of this repo. We are happy to discuss this potential application with you, but maybe at some other venue. I would suggest you to check related literature first. My previous response was mainly to convey the idea that ViT can be used for processing images in general.

Directly using VIT to process video may result in bad results.