Stable Diffusion General utilities wrapper including: Video to Video, Image to Image, Template Prompt Generation system and more, for use with any stable diffusion model.
Note, this is by far not a finished technology and it will be continously improved upon. This was kept as modular as possible save for including torch code in the genutils.generate
function.
- Image2Image generation witih seed control
- Video2Video generation with seed control and overrides
- Multiple Image to Multiple Image generation
- Batch Processing for Image2Image and Video2Video - Get as many as you want from one input
- Outputs all seeds, strengths and prompts generated into a file called vidmap.json
- Stores all output videos and images with seed data and index number for easy association with stored map
- Keyframes for video2video and prompt changes
- Image/Video in-painting and out-painting
- NFT Listing Capabilities for OpeanSea perhaps?
PromptGenerator
generates prompts based on possibilities you give it so that you can easily change out any word or phrase in your prompt with a random entry from a list of possibilities. It makes prompt templates re-usable, just change the data to get different prompts
Scaffold
holds all the util functions and takes a (device, scheduler, pipe) from the torch environment thus wrapping the initial Stable Diffusion functionality and utility functions into one class, and expanding on that functionality with a Video2Video function, and batching
Takes just a map of possible attributes and a prompt template including those attribute variables into a prompt.
# make sure to have your device, scheduler and pipe declare in your torch code, then put them in the scaffold ...
scaffold = Scaffold(device, scheduler, pipe)
Setting up a new Prompt with different possible outputs
map = {
'attribute1': ['tall', 'short', 'lumpy'],
'attribute2': ['headed', 'framed', 'legged'],
...
}
prompt = PromptGenerator('A self portrait of a $attribute1 $attribute2 person', map)
What about getting a text prompt from this?
text_prompt = prompt.generate()
# would output from above example:
# A self portrait of a tall legged person
# A self portrait of a lumpy headed person
# ...
# for every time you call the generate function you get a different result based on values from the map
img_path = 'path_to_some_image.jpg'
scaffold.generate(
img_path,
prompt,
num_seeds = 1
)
scaffold.generate(
img_path,
prompt,
num_seeds = 100
)
video_path = 'path_to_some_video.mp4'
map = { ... }
prompt = PromptGenerator( ... )
scaffold.generate_video(
video_path,
prompt
)
# Generate 10 different videos from one video
scaffold.generate_batch_videos(
video_path,
10,
prompt
)