How to train this on a custom dataset ?

[Damien Menigaux]

I've been looking at this for the past few days and I can't seem to figure out how I can use this to train on any other dataset than Thumos or ActivityNet. How would you train this in the end-to-end fashion proposed in the article please ?

[guancheng817]

same question..

[95xueqian]

@hyperfraise have you know how to train this model on other dataset? thanks

[Damien Menigaux]

I have, I also added a classification part, and made it really end-to-end. I'm not sure t's the most proper way of doing it but I'll show you soon.

[guancheng817]

I have, I also added a classification part, and made it really end-to-end. I'm not sure t's the most proper way of doing it but I'll show you soon.

Thanks for your works, look forward to your open source..

[Damien Menigaux]

Here you have my work. I welcome criticism and error pointing !

https://github.com/hyperfraise/end-to-end-BSN.pytorch

[shawxiao]

Hello, I would like to ask what kind of input data you have implemented for end-to-end BSN. I don't understand some of your source code. I'm sorry!

[Damien Menigaux]

Videos of such shape (batch_size, channels, video_length, image_size, image_size)

[Damien Menigaux]

And labels of shape [class, t_start, t_end]

[Damien Menigaux]

See the https://github.com/hyperfraise/end-to-end-BSN.pytorch/blob/master/example.py file for demo on how to use it.

[shawxiao]

Is there any script file processed? Take the thmos2014 data set as an example.

…

------------------ 原始邮件 ------------------ 发件人: "hyperfraise"<[e-mail]>; 发送时间: 2019年7月18日(星期四) 晚上10:52 收件人: "wzmsltw/BSN-boundary-sensitive-network"<[e-mail]>; 抄送: "摸摸头^^"<[e-mail]>;"Comment"<[e-mail]>; 主题: Re: [wzmsltw/BSN-boundary-sensitive-network] How to train this on acustom dataset ? (#35) Videos of such shape (batch_size, channels, video_length, image_size, image_size) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[Damien Menigaux]

Custom datasets will have the particularity of having custom dataloader and custom formats of data storage, label storage etc... So I thought that the only thing I could actually do is tell the reader the format of the data that should be passed to the loss functions, and let the reader decide how (s)he needed to transform his(er) jsons and videos to this end.

However, you're right that it would allow for more flexibility on Thumos14 architectures. I'll try to implement a simple dataloader for this as soon as possible.

[shawxiao]

Thank you very much.

[shawxiao]

I would like to ask your model features or the features of brother tianwei. The video feature extraction was not added to the end-to-end architecture.

…

------------------ 原始邮件 ------------------ 发件人: "hyperfraise"; 发送时间: 2019年7月18日(星期四) 晚上10:53 收件人: "wzmsltw/BSN-boundary-sensitive-network"; 抄送: "shawxiao"<[e-mail]>;"Comment"; 主题: Re: [wzmsltw/BSN-boundary-sensitive-network] How to train this on acustom dataset ? (#35) And labels of shape [class, t_start, t_end] &mdash; You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[Damien Menigaux]

Hey. i'll respond to issues on my code, on the aforementionned repo. If you've got suggestions, I'll be glad to adress them there. I don't think here is the best place :)

https://github.com/hyperfraise/end-to-end-BSN.pytorch

[Malithi-gif]

@hyperfraise have you apply this into custom video dataset?

[Damien Menigaux]

Yes I failed to go above baseline accuracy. But I never could with temporal detection framework anyways :/

[Malithi-gif]

@hyperfraise, Thanks for your reply. What method use for get the video features? TSN?

[Damien Menigaux]

I trained it fully end to end with 2d resnet 18 feature extractor with the code I linked. So I extracted features on the fly.