[Bug]: Segmentation generated behavior not as expected

Question

[Bug]: Segmentation generated behavior not as expected

alexknows opened this issue 2 months ago · comments

Is there an existing issue for this?

I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I recently came across OlivioTutorials video on ControlNet's segmentation, but it seems that the generated images is not adhering to it.

Steps:

I generated the map in the ControlNet's section.
I Downloaded map and made a few small tweaks in Photoshop. (e.g to remove a piece of wall and an object)
I replaced the original with the map.
I added the prompts and adjusted all the settings accordingly.
The image it generates has no similarity what so ever to the map I provided in ControlNet.

It seems as if ControlNet is not even enabled (but it is). No errors are thrown.

Steps to reproduce the problem

I generated the map in the ControlNet's section using an image of a bathroom.
I Downloaded map and made a few small tweaks in Photoshop. (e.g to remove a piece of wall and an object)
I replaced the original with the map.
I added the prompts and adjusted all the settings accordingly.
The image it generates has no similarity what so ever to the map I provided in ControlNet.

First, I added the image to controlnet:

I generated the following map:

I saved the map and made minor tweaks to it in photoshop:

I added the prompt:
RAW photo, a modern bathroom with a tub, sink, toilet, center area with a wooden two-tone flooring tile, a jigsaw puzzle style flooring, isometric view, (high detailed), 8k uhd, high quality

Negative prompt
semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, duplicates

Adjusted all the controls: