The training and testing (loading and using) of a model should be in the same class or docs should be added
futurely opened this issue · comments
The classes in the gr.iti.mklab.visual.quantization
package train the various models used by the classes in the gr.iti.mklab.visual.datastructures
package. It's hard to construct the data pipeline of training and testing the same complete search structure.
I think that it's better to distinguish classes that implement data structures from classes that train the quantizers used from this data structures. That said, adding documentation and/or a class that shows how the complete pipeline (of training the quantizers and using them in a data structure) can be constructed would indeed be very helpful.
CoarseQuantizerLearning and CodebookLearning are essentially the same except for the input data format. They can be unified.
I've successfully built a small IVFPQ index and the image retrieval results were not bad although I didn't benchmark and tune the parameters. The complete steps are as follows.
- Downloading images (using my own Apache HttpComponents based multi-threaded downloader)
- Run
gr.iti.mklab.visual.examples.FeatureExtraction
to extract local features. A few file path issues have to be fixed. SIFT and RootSIFT failed to detect key points in some of my images while SURF worked well. - Optionally sample the local features by running
gr.iti.mklab.visual.quantization.SampleLocalFeatures
. 4. Copy the local features of each image into a single file withcat feature_dir/*.txt > somewhere_else/features.csv
orcat feature_dir/*.csv > somewhere_else/features.csv
. - Run
gr.iti.mklab.visual.quantization.CodebookLearning
to learn a local features codebook. - Run
gr.iti.mklab.visual.examples.FolderIndexingMT
to extract VLAD features of the images downloaded in step 1 using the codebook learned in step 5 and store them in a linear index. - Run
gr.iti.mklab.visual.quantization.CoarseQuantizerLearning
to learn a coarse quantizer from the VLAD features extracted in step 6. - Run
gr.iti.mklab.visual.quantization.ProductQuantizationLearning
to learn a product quantizer from the VLAD features extracted in step 6 and the coarse quantizer learned in step 7. - Run
gr.iti.mklab.visual.examples.IndexTransformation
to transform the linear index of the VLAD features extracted in step 6 into an IVFPQ index using the product quantizer learned in step 8 and the coarse quantizer learned in step 7. - Run the image retrieval example
gr.iti.mklab.visual.examples.Example
using the linear index (step 6), IVFPQ index (step 9), coarse quantizer (step 7) and product quantizer (step 8) to get the most similar images of each image in the linear index. I adapted the example to generate the HTML pages each of which visualized a query image and its retrieval results.
The pipeline seems correct.