VLB: VLFeat Local Features Benchmark
A framework for local feature evaluation. Reimplementation of the VLBenchmarks project. Currenlty work in progress so it has more than few rough edges.
MATLAB implementation: Karel Lenc
Python implementation: Xu Zhang
Getting Started with MATLAB implementation
Setup matlab mex environment, before starting to setup vlb.
To setup the VLBenchmarks, run from the vlb path:
run ./matlab/vlb_setup.m
run ./matlab/vlb_compile.m
where one needs to compile the package only while running for the first time.
For example usage, see the prepared examples:
- detbench.m - Detector benchmarks and results visualsiation
- descbench.m - Descriptor benchmarks
Basic Usage
The entry function is vlb which supports the following commands:
FEATSNAME = vlb('detect', IMDBNAME, DETNAME)
Detects features of all images of a IMDBNAME image database. Features stored
./data/features/IMDBNAME/DETNAME/ as csv files.
Additionally one can also extract the detected patches with the extract command.
FEATSNAME = vlb('describe', IMDBNAME, FEATSNAME, DESCNAME)
Computes descriptors of all features detected by FEATSNAME of IMDBNAME image database.
Features stored in ./data/features/IMDBNAME/FEATSNAME/DESCNAME/ as csv files.
RES = vlb('detrep', IMDBNAME, FEATSNAME)
RES = vlb('dematch', IMDBNAME, FEATSNAME)
RES = vlb('descmatch', IMDBNAME, FEATSNAME)
Evaluates the detector repeatabiliy, detector matching score or a descriptor matching score of features FEATSNAME in the selected dataset. Returns a table with results,
which is additionally stored in ./data/scores/BENCH/IMDBNAME/FEATSNAME as a csv file.
Custom Features
For custom features, you can either add your own wrapper to matlab/+features/+det or matlab/+features/+desc, depending whether the input is an image or patches/detected keypoints. Afterwards, you can use the vlb interface directly.
However, you can also compute the features outside of MATLAB and store them as CSV files. To get the list of image paths and target feature paths in format: IMAGEPATH;TARGET_FRAMESPATH;TARGET_DESCSPATH you can run:
vlb imagelist IMDBNAME FEATSNAME
Typically, this would create a file in data/IMDBNAME-FEATSNAME.csv
The format of the CSV files is one descriptor/frame per line. The frames or one-indexed (center of the first image pixel has a coordinate [1, 1]) and in format of vl_plotframe ([c, r] for a point, [c, r, s] for a disc, [c, r, s, rot] for oriented disc, [c, r, S11, S12, S22] for ellipse, [c, r, A11, A12, A21, A22] for oriented ellipse/affine frame).
Directory structure
data/features/DATASET/FEATURES Compputed local features in csv format (see vlb detect and vlb describe)
data/patches/DATASET/FEATURES Extracted patches as png images (see vlb extract)
data/scores/BENCHMARK/DATASET/FEATURES Computed benchmark scores, stored as .csv tables
datasets Downloaded datasets
matlab MATLAB source code
matlab/+dset Implemented datasets (VGG-Aff, EdgeFoci, WebCam, Hsequences...).
matlab/+features/+det Implemented detector wrappers.
matlab/+features/+desc Implemented descriptor wrappers. Please note that some descriptors accept detected frames, some need extracted patches.
Furthermore, you can extract the features outside of this framework in order to evaluate them. Please extract e.g. vlsift features and follow the naming conventions.
Compile Conflicts
mex command may be recognized as mex command in texlive. Make sure that the matlab/bin path is added to system PATH before the directory of mex in texlive.
Running the tests
To run the tests, call:
vlb_test.
Python Interface
Requirement
Make a complete clone operation to pull the submodule.
git clone --recursive git@github.com:lenck/vlb.git
We recommend to use conda to install all the requirements all at once.
conda env create -f ./python/conda/environment.yml
source activate vlb
pip install nmslib
Test the code
Test repeatability benchmark
python ./python/test/test_rep_bench.py
Test matching score benchmark
Download vlsift_load_matlab.zip and extract it to ./data/features/vggh/. Then run
python ./python/test/test_ms_bench.py
Test image retrieval benchmark
python ./python/test/test_retrieval_bench.py
Test wide baseline matching benchmark
python ./python/test/test_W1BS_Bench.py
Test feature extraction
python ./python/test/test_feature_extraction.py
Test draw feature
python ./python/test/draw_frame.py