This repository powers an interactive visual embedding browser for my drawing portfolio at projector.andrewlook.com.
I adapted this repo to include my own data, and also to modify the underlying GUI code. Specifically, the image view in the left-hand inspector panel was available in the official tensorboard release inside of vz-projector-inspector-panel.ts and vz-projector-inspector-panel.html.ts, but not in the compiled standalone version.
This repository is based on TSNE-UMAP-Embedding-Visualisation by Harshvardhan Gupta, which is a better-documented version of Tensorflow's Standalone Embedding Projector. The underlying code (not standalone) lives in the tensorflow repo under tensorboard/plugins/projector.
It's preloaded with about 1900 single-line drawings scanned from my sketchbooks:
|-- index.html <-- The GUI of the Viewer
|-- oss_data <-- published assets used to load data into the GUI
| |-- oss_demo_projector_config.json <-- configuration for embedding projector, same format as projector_config.pbtxt
| |-- sprite.jpeg <-- all source images arrayed into a single jpeg
| `-- feature_vecs.tsv <-- tab-delimited embeddings
`-- projector_config.pbtxt <-- configuration to run embedding projector via tensorboard CLI (not standalone)
From the project root, run the following command and then navigate to http://localhost:8000:
python3 -m http.serverTo compare this standalone, hostable version the official tensorboard implementation, run the following and navigate to http://localhost:6006/#projector
pip install -Uqq tensorboard
tensorboard --logdir .Visualizing Data using the Embedding Projector in TensorBoard from the official tensorflow documentation gives a helpful introduction, but doesn't cover image sprites.
Taking the TensorBoard Embedding Projector to the Next Level by Ángel Igareta goes into more depth, specifically covering how to put together the large image sprite.
A stackoverflow answer clarified how to add multiple metadata columns (alongside the label): python - Tensorboard - Adding multiple metadata labels to data points - Stack Overflow.
- How to Use t-SNE Effectively
- How UMAP Works — umap 0.5 documentation
- [1802.03426] UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- machine learning - How to understand the drawbacks of K-means - Cross Validated
Ampligraph has a cool utility, create_tensorboard_visualizations to streamline creation of the metadata/configs needed to power the embedding projector.
Machine Learning for Artists ("ml4a") published some useful jupyter notebooks and guides for rendering sets of images as 2D TSNE images. Specifically, there's a 2D Grid Image TSNE using RasterFairy to arrange the pointcloud into a 2D grid.
From RasterFairy's description:
The purpose of Raster Fairy is to transform any kind of 2D point cloud into a regular raster whilst trying to preserve the neighborhood relations that were present in the original cloud. A typical use case is if you have a similarity clustering of images and want to show the images in a regular table structure.
The RasterFairy README has a gif that summarizes this well:
Ml4a also provides some examples of a standalone UI desktop app, using the OpenFrameworks C++ framework to make an interactive, browsable experience in a native application: