The goal of this repo is to modify
pix-plot to work as a front
end for exploring any image embeddings, for example those from
clip_retrieval.
If you want to go directly from a folder of JPGs to a PixPlot
visualization, you should use
pix-plot itself.
This is merely a work-in-progress attempt to enable you to generate a PixPlot visualization after creating embeddings.
Why do this? 1. clip gives useful embeddings as shown by the first
place
solution
in the recent “Google Universal Image Embeddings” Kaggle competition 2.
Separating the embeddings step gives more flexibility in how it is run –
for example, it can be run in parallel on several nodes 3. Separating
the embeddings step also makes it easier to remove pinned dependencies
so the exciting viz stuff is easier to install
Initial ambitions coming from PixPlot:
- ☑️ Unpin requirements versions
- ☑️ Remove all TensorFlow dependencies
- ☑️ Enable to go directly from what you’d expect to have after running
clip-retrievalinference: i.e. a folder of JPGs and a folder with embeddings in numpy format
Later ambitions
- ⬛ Add a linear dimensionality reduction method to complement UMAP
- ⬛ Include option to output as a desktop app via Tauri
- ☑️ BYO logo
clip-plot is a fork of
pix-plot by Douglas
Duhaime, Peter
Leonard, and others, and is mainly a
lighter, smaller version of their work.
The DHLab would like to thank Cyril Diagne and Nicolas Barradeau, lead developers of the spectacular Google Arts Experiments TSNE viewer, for generously sharing ideas on optimization techniques used in this viewer, and Lillianna Marie for naming this viewer PixPlot.