Possibility to Change the Text Encoder?

Question

Possibility to Change the Text Encoder?

mcairlangga2 opened this issue a year ago · comments

Muhammad Cendekia Airlangga commented a year ago

Dear Authors,
Thank you for the great work.

I want to ask a question. Is it possible to change the text encoder to other models such as CLIP instead of using RoBERTa? Have you considered and tried another Text Encoder? If it's possible how to change it in the code?

Thank you!

Muhammad Maaz · Answer 1 · Sat Jun 03 2023 05:59:01 GMT+0800 (China Standard Time)

Hi @mcairlangga2,

Yes, it is possible however we did not consider this research direction. With the recent advancements in NLP & Vision-Language modeling, it will be a worth exploring problem.

As per the implementation is concerned, it would be replacing RoberTa at this file with CLIP or other text decoder. Good Luck