JuliaImages / ImageContrastAdjustment.jl

A Julia package for enhancing and manipulating image contrast.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Generalize `LinearStretching`

stillyslalom opened this issue · comments

There's significant overlap between LinearStretching and a family of related, non-deprecated functions in ImageCore (https://juliaimages.org/latest/function_reference/#ImageCore.scaleminmax, etc.). However, the overlap isn't complete (there's no analogue here for scalesigned(min, center, max), and the APIs don't match, which may become a source of confusion. To reduce the size of the API surface, the missing methods should be added to ImageContrastAdjustment, and the corresponding functions in ImageCore should be deprecated.

To incorporate scalesigned(min, center, max) and scalesigned(maxabs) into LinearStretching, the source -> target intensity mapping can be generalized to accept three (or more) intensity levels, representing a piecewise-linear intensity transformation. Johnny's recent PR #28 is a good base (thanks!), but the function and interface would need to be further overhauled for piecewise-linear maps.

I'd also like to cover the use-case described by @bjarthur in issue #27:

My sole use case is imadjustintensity(img, quantile(img,[0.01,0.99])), which i find quite good at eliminating outliers when adjusting the brightness.

I've typed out the exact same function call countless times in various projects over the last few years. For a more Julian interface, let's look to Percentile. For example,

  • LinearStretching(Percentiles(1, 99))
  • LinearStretching(Quantiles(.015, .985))
  • LinearStretching(Percentiles(0, 50, 100), (-1, 0, 1)) would replace scalesigned(min, center, max)
  • LinearStretching(maxabs, (-1, 0, 1)) would replace scalesigned(maxabs)

To add a cherry on top, let's add some convenience methods. Matlab has imadjust, we once had imadjustintensity, and now we have

adjust_histogram(img, LinearStretching(quantile(vec(img), (0.01, 0.99) => nothing))

While that's great for clarity, it's bad for usability. Assuming my proposed refactoring #32 goes through, we could dispatch on ::Quantile from adjust_intensity:

  • adjust_intensity(img) # defaults to extrema
  • adjust_intensity(img, Percentiles(1, 99)) covers @bjarthur's use case

I think a new PiecewiseLinearStretching type would be more suitable for the more general case of specifying an arbitrary number of stretching operations over user-specified intervals. However, I'm not sure that evenPiecewiseLinearStretching is necessarily the right replacement for scalesigned(min, center, max) etc. Those functions seem to be more "low-level" in the sense that they appear to be useful in a context quite apart from just enhancing the contrast of an image. I haven't used them myself, but perhaps @timholy can comment on what context they are used most frequently. The contrast enhancing operations in ImageContrastAdjustment are in principle supposed to return a valid image with values in the unit interval. A further convention used in this package is that all operations on color images are applied to the Y channel of the YIQ colorspace, and the result is transformed back to whatever colorspace the user passed in. My understanding is that the linear stretching operations in ImageCore are applied separately on each color channel.

I think the idea suggested in #32 of introducing
AbstractIntensityMappingAlgorithm <: AbstractImageFilter and associating LinearStretching, ConstrastStretching, GammaCorrection is reasonable, and I would be willing to put in the time to make the necessary changes and introduce a adjust_intensity function to cater for those types if we can reach consensus on the suggestion.

However, I'm not sure that I support the suggestion of having adjust_histogram and adjust_intensity by default privilege any one particular algorithm. It looks like all the quibble is centred on the fact that a very commonly used operation (linear stretching based on percentiles) is long to type, and not a pleasant experience when doing exploratory work on the REPL. I think that if this is indeed such a commonly used operation it might be better to have a bespoke alias for it, such as span_contrast with all the lovely dispatch behaviour on percentiles, quantiles, AbstractRanges.