BriVL for application development

This repo is used for illustrating how to build applications by using BriVL model.

This repo is re-implemented from following projects:

Online Demo built by BriVL

Test this Pipeline

Test image has been saved in ./bbox_extractor/feature_extractor, test with following command:

python3 main.py --brivl_cfg BriVL/cfg/BriVL_cfg.yml --brivl_weights BriVL/weights/brivl-weights.pth

Download Models

bua-caffe-frcn-r101_with_attributes.pth -> /bbox_extractor/weights
chinese-roberta-wwm-ext -> /BriVL/weights/hfl
tf_efficientnet_b5_ns-6f26d0cf.pth -> /BriVL/weights
brivl-weights.pth* -> /BriVL/weights

Requirements

Python >= 3.6
PyTorch >= 1.4
Cuda >= 9.2 and cuDNN
Detectron2 <= 0.3
Transformers

Important: The version of Detectron2 should be 0.3 or below.

Install Pre-Built Detectron2 (Linux only)

Choose from this table to install v0.3 (Nov 2020):

CUDA	torch 1.7	torch 1.6	torch 1.5
11.0	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html`
10.2	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.6/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.5/index.html`
10.1	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.7/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.6/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html`
9.2	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu92/torch1.7/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu92/torch1.6/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cu92/torch1.5/index.html`
cpu	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cpu/torch1.7/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cpu/torch1.6/index.html`	install `python -m pip install detectron2 -f \ https://dl.fbaipublicfiles.com/detectron2/wheels/cpu/torch1.5/index.html`