ABTester is a comparative tool designed for evaluating sets of images by allowing users to select their favorites through a straightforward interface. This utility is especially useful for subjective analysis of image sets, including but not limited to assessing the output of different epochs or versions of an AI model. By facilitating an unbiased preference selection process, ABTester helps identify the most appealing images within a dataset according to the user's perspective.
It is always recommended to set up a Virtual Environment first. Follow the instructions there, or create one yourself.
Install the requirements from requirements.txt using pip install -r requirements.txt.
Run launcher.bat, or py app.py.
If successful, it should launch the flask server, and the website, and it should ask you the number of images in each set. To test it out, enter 4 here.
To prepare your images for testing, place them in the /static/IMAGES/ directory. The tool supports both PNG and JPG formats. Image loading is sequential; thus, the naming convention of the files is significant for the testing sequence. Here is the expected naming format to organize image sets:
01.jpgfor Set 102.jpgfor Set 203.jpgfor Set 304.jpgfor Set 405.jpgreturns to Set 106.jpgfor Set 2, and so forth, following the pattern: 1, 2, 3, 4, 1, 2, 3, 4, etc.
This naming scheme ensures that images are grouped and presented in sets for comparison.
To initiate the comparison tool:
- Execute the
launcher.pyscript. This action starts a Flask server and opens the tool in your web browser. - Upon reaching the landing page, input the total number of image sets you're comparing. This number should correspond to the grouping sequence established in your image setup.
- Utilize the interface buttons to navigate through the testing process. Options include restarting the test, toggling set number visibility, and revealing the aggregated results of your selections.
The tool's design ensures a user-friendly experience for efficiently conducting image preference tests.
- 2024-05-10
- Can now maintain a shuffled set without getting the same set twice until you have voted on each image.
- Added option to skip voting.
- Now displays the current voting round values at the bottom.
- Fix bug with the viewing starting at the wrong index the first round of voting.
- Add display of EXIF metadata (optional type, like prompt)
- Add a way to remove a set of images from the dataset. It should move them to a 'removed' folder.
- Add option on the index page to choose mode.
- Single Vote (as it is now)
- Multi Vote (multiple images can be voted on, manually go to the next vote).
- Last Image Standing (#1)