EmbarkStudios / texture-synthesis

🎨 Example-based texture synthesis written in Rust 🦀

Home Page:http://embark.rs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Proper alpha channel support in matcher.

vi opened this issue · comments

Currently alpha channel is ignored in matcher algorithm. It is used just in final phase, leading to illusion of alpha channel support.


Some comments by @austinjones from #22:

@vi @Jake-Shadle I have a prototype of generic pixel support (RGB/RGBA/Luma/LumaA) implemented on austinjones/texture-synthesis/multipixel, but it needs some work. It's just a quick exploration.

I tried handling alpha this way: Premultiply alpha and only consider color channels. This makes sense when later compositing: bright red with very low alpha and bright green with very low alpha are actually similar colors when added to anything. I didn't add alpha to the cost computation though, because the correlation with the color channels will cause bias in the cost metric.

There were some side effects:

* Session requires a type parameter (the pixel type).  It has to be specified by the user whenever SessionBuilder::build() is called.

* There is a decent performance hit.  Need to figure out why...

* The guided synthesis test fails.

Let me know if you are interested in the code. I can try to fix it up.


@vi Yes, you are right! I was trying to solve a different problem (support for example transparency).

I think what is supported right now is cost computation based on RGB, but pixel copies are on the full RGBA. If you add a bunch of transparent bright red, the cost is computed on bright red. Take a look at k_neighs_to_color_pattern - note the multiplier on the first line is 3, not 4. Transparent bright red is considered bright red when running the matching algorithm.

As for me, alpha channel can serve as:

  • Something like a 4-th colour channel, in addition of red,green,blue (if algorithm does not use YUV).
  • "Importance weight" for non-alpha channels - the more transparent area is, the less important it is. Fully transparent => weight=0 => no effect on matcher. Obviously, this weight does not apply to alpha channel content itself.