srkn / stable-diffusion-sqlite

All Stable Diffusion images in a sqlite database

SQLite DB of all Stable Diffusion generated images

Not too long ago, Turquoit#3987 from the Stable Diffusion Discord server shared a text file containing links to every image generated by the bot from August 6th (its inception) to approximately August 15, 2022. This repository contains some code to convert that file to a SQLite database for easy querying. A prebuilt ImageLinks.db file is provided here (3.2gb)

Structure of the database is two tables:

• prompts, with a primary key id and a string prompt
• data which has a promptId and a url, among extra metadata columns

The converter is written using Node.js and better-sqlite3, and took 162 seconds to convert on my desktop.

There is also an implementation with Bun, but it doesn't work for me due to a bug. Were it to work, it would run over 3x faster than the node implementation.

To run, you need an extracted copy of ImageLinks.zip (2.5gb). Then you can run bun convert.ts or node convert.mjs.

Maybe in the future, a web app / public web api will be made to query the database without storing a local copy. I could do this but I wouldn't be able to fund a server to hold the database (ideally use postgres instead of sqlite).

Changelog

2022-08-20

• Add indexes to the SQLite database, decreasing query times of "all images for a prompt id" from 8000ms to 3ms on my machine

2022-08-19

• Initial Publication