Resizing from <img> element results in broken output in Chrome 81
FarSeeing opened this issue · comments
Hi. First of all - many thanks for that library!
We've been using it quite a lot but recently faced several images that result in broken output when using Chrome and <img> element as source.
If used in Firefox - everything's fine.
If used in Chrome with <canvas> element as source (without any transformation, just drawing image) - everything's fine. Options and features don't affect this.
Original image (one of) - https://dl.dropboxusercontent.com/1/view/zjnu3kgtyzq6zv3/pica/original.jpg
Corrupted result (<img> source) - https://dl.dropboxusercontent.com/1/view/c6s817ypu7uibgi/pica/from%20image.png
Correct result (<canvas> source) - https://dl.dropboxusercontent.com/1/view/kh4cisii050o4gn/pica/from%20canvas.png
Playground for testing - https://codepen.io/msidorov/full/PoPdVjV
All these were taken from Chrome 81, different OSes
Many thanks in advance.
Isn't this dupe of #190?
I don't think so - no error is thrown and createImageBitmap returns data
I've published fix for #190.
What about this one - probably they've broken something more serious in chrome 81. Is this still a problem or they fixed?
I updated today to latest 5.1.1 but still have a problem at chrome (81):
Resized an image in FireFox:
Resized same image in Chrome:
(No matter about orientation, thats just another strange thing which i can handle with the exif orientation i think)
@puzrin thanks for update. Unfortunately it doesn't help - createImageBitmap doesn't throw error so code in catch block is ignored.
Does problem happen with any image? Seems they have broken createImageBitmap and this should be reported to chromium tracker. I have no technical possibility to detect correctness of createImageBitmap :(.
Disabling use of createImageBitmap it is not nice, because it can unpack image in background and prevent main thread freeze.
PS. I hope you did NOT enabled CIB option, it's not tested at all and has bad quality in chrome.
For me: No, i did not enable CIB
const resizedCanvas = await pica.resize(image, canvas, { unsharpAmount: 80, unsharpRadius: 0.6, unsharpThreshold: 2 });
And no it does not happen with any image. At the moment i only have the example from above cause if i try another one in Chrome:
Here is the original file which makes trouble: https://transfer.sh/Z7OxF/IMG_2448.jpg
Same here - no, I'm not using cib (you can check the whole options set here) and no, that doesn't happen on every image.
BTW I also have EXIF issues in Chrome as @CSoellinger-IDS mentioned but I'm not sure if that's related.
EXIF orientation processing is a separate story. Pica does not use it anyhow. Proper image file resize need more care, i refered my code sample in some prev issues. That's a lot of code and currenly out of this package scope. This package is about raw bitmap objects, not about files of specific format.
So, let's keep orientation aside. We can continue about that in prev issues, but i really don't know how to put that in current api.
Let's focus on creating reproducible bug demo (for report to chromium tracker). I currently could not reproduce problem in https://nodeca.github.io/pica/demo/ with Chromium 81.0.4044.138, Ubuntu 18.04.
Need help with samples:
- How to reproduce (via https://nodeca.github.io/pica/demo/ or via your test repo)
- Some stats - does this happens often or not (% of images or anything you think useful)
@puzrin - you won't be able to reproduce issue on that demo page because it uses canvas as image source while you need img
A bit of demo script:
resizer.resize($('#src')[0], offScreenCanvas, {
where $('#src')[0] is canvas used to display original image.
Sorry if that looks like I'm promoting my work but I've created simplified playground where issue is visible - https://codepen.io/msidorov/pen/PoPdVjV
Regarding second question - sorry once again, I don't have even a slightest estimation on % of broken images. It was working perfect for quite a long time and still works for most of images but also we faced a bunch of images that are broken (and we don't have a clue why).
Thanks, now reproducible.
So, current steps for reproduce:
- Go to https://codepen.io/msidorov/pen/PoPdVjV
- Use attached file to see error when
createImageBitmapused
One more question. Could you add here return null (force fallback) and tell me if it helps or not? Need to understand, is problem related to createImageBitmap indeed, or any partial extract from image affected.
Tried this (returning early with null) - same result.
Hm... then problem should be at this line: https://github.com/nodeca/pica/blob/master/index.js#L368, and should be reproducible without pica at all.
Try to draw from [buggy] image to canvas the same regions, like pica does, and you should see problem with some. That will be minimal possible sample, easy to understand for chromium devs.
For sure - disable webworkers. Concurrency should no be an issue, but... just for sure.
PS. sorry that i ask you do something, i'm not "developper for browser" at all. Most of my experience consists of node modules browserification :)
Sorry, I don't have a minimal example now, will try to create it today.
What I've found so far:
- WebWorkers, concurrency and alpha don't affect issue,
- tile size - affects partially: if I set tile = image max dimension then everything's fine.
Absolutely ok for helping you to debug it (since I'm interested in solving it).
If you have any questions about pica sources - let me know.
IMO, this steps required:
- dump to console tiles list (array of area coordinates)
- try to set output image as max as possible, to avoid multi-step downsample (don't resize 10000*10000 to 10*10).
- apply draw those coords to canvas and check result.
Optionally - do the same in parallel for image & canvas, and compare results to automate things. But probably you will succeed with 1-2 hardcoded regions.
If succeed - create a test repo or anything permanent, and this can be reported to chromium tracker, as "Draw region from jpeg to canvas, broken data, regression in v81". When simple and easy to reproduce sample exists, chromium guys should fix fast.
Ok, so I found it.
Chrome 81 added image-orientation CSS property with default value from-image meaning it will take EXIF orientation from image and will display image properly. Developers can disable that by setting image-orientation: none that I applied.
Now the weird part - the image that was failing is rotated 180° so the first tile that should start in upper left corner (in "proper" coordinates) starts in lower right corner (again in "proper" coordinates) but the tile itself is rotated.
Could be difficult to understand at first so here's the scheme:
I'm not sure but it seems to be a bug in Chrome or some weird decision that maybe controlled somehow (although not obviously)
What helps for the moment is creating a canvas... doing it like this:
export default class Canvas {
static async fromImage(image) {
const canvas = document.createElement('canvas');
canvas.width = image.width;
canvas.height = image.height;
canvas.getContext('2d').drawImage(image, 0, 0);
return canvas;
}
static async toImage(canvas) {
const image = new Image();
image.src = canvas.toDataURL('image/png');
return image;
}
}
Then resize:
const canvasImage = await Canvas.fromImage(image);
const canvas = document.createElement('canvas');
canvas.width = calculateImageSizes.width;
canvas.height = calculateImageSizes.height;
const resizedCanvas = await pica.resize(canvasImage, canvas, {
unsharpAmount: 100,
unsharpRadius: 1,
unsharpThreshold: 1,
});
Getting this result:
Guys, I'm closing this issue since that's not the library case, it's already been reported to Chromium tam way before me - https://bugs.chromium.org/p/chromium/issues/detail?id=1074354
@FarSeeing thank you for investigation and details. Today received Chrome 83.0.4103.61 in update (from Google). Seems to work. Are your setups autofixed?
Ubuntu's Chromium (NOT Chrome!) still 81.0.4044.138, but i think will be updated soon.
BTW, may be someone has time to kick chrome devs about #89 (see link inside). Problem is, this feature was released with bad image quality, and blocks it's usage. I succeeded to prove problem in chromium tracker. But ticket was postponed multiple times and silently trashed.
As i said, i'm not "dev for browser", and can't focus for a long time on such kind of activities. It stays open as reminder, but without move forward. If any help about technical details needed - i'm always here.
@puzrin - that issue was indeed fixed version 83 but it raised another issue - now drawImage in Chrome returns data for properly rotated image while other browsers still return it "as is" (rotated) and now I need to make a workaround for that but that's still a step forward.
Regarding other issue - alas I don't have direct contact to Chromium/Chrome devs so the only thing I can do (and I did) is to star the issue.
but it raised another issue - now
drawImagein Chrome returns data for properly rotated image while other browsers still return it "as is" (rotated) and now I need to make a workaround for that but that's still a step forward.
IMO this is NOT about drawImage only. As far as i understand ALL image properties should be emulated (width, height) to work as with rotated image.
So, resize should be ok, BUT problem is it unsyncs with EXIF processing logic. I'm not sure this should be handled in pica, but it certaintly should be handled somehow.
If you do EXIF processing, I see 2 possible solutions:
- Intruding. Force
image-orientation: noneand leave everything else as is. If you create image from file and not use it anywhere - that's ok. - Use special 1*2px image to detect if
image-orientationsupported and active, and drop orientation from EXIF.
What about pica - IMO it would be enougth to add warning into readme, "If you do additional EXIF-based processing, make set image-orientation: none for input image objects to avoid conflicts". May be this needs better sample for correct fallback to other browsers.
Let me know what do you think.
Just for reference: https://github.com/nodeca/nodeca.users/blob/master/client/users/uploader/uploader.js
There is example how images are processed in my project (it rebuilds and transfer EXIF-s). Including intelligent streamed parse & tags strip. It's a bit sophisticated, but i could not invent anything more simple without side effects.
In ideal world, that should be arranged to standalone package (wrapper around pica), specially for blobs (files) resize. This did not happened because lack of motivation. Until no commercial companies offered anything interesting, i suggest anyone to reuse those sources "as is" and "as you wish" (MIT). I'm not greedy about my code :).
Well, that's the issue - image-orientation doesn't work for canvas for some reason.
Even if you set image-orientation: none canvas functions will still behave wrong (yep, I may be really-really wrong here but quick test reveals that).
So yes, that are solutions but they all come to double-processing image.
I use similar uploader and it does these steps:
- load file into url,
- process file to extract EXIF orientation,
- resize image with
pica - rotate resulting canvas based on extracted orientation (if needed).
Now it seems that I will have to add one step before resizing. Not critical but still not perfect.
@FarSeeing could you review API of this draft https://github.com/nodeca/nodeca.users/tree/filterjpeg2/lib/image_blob_reduce ?
I'd like to release it as standalone module (and micrate my apps to it). Is that ok for your needs or something missed?
PS. Internals (jpeg plugins) are not finished yet, but this should work in couple of days.
@puzrin - I haven't checked that app "in the wild" but it examined that code and (here I may be totally wrong) it doesn't work around Chrome orientation issue.
What is that issue: Chrome 83 fixed canvas method of drawing images but it's totally different from Firefox in the way of treating EXIF orientation: while FF returns original (not rotated) image, Chrome tires to rotate it. Thus you just get some image but don't really know if you should or should not rotate it.
I've managed to solve this buy first removing orientation tag at all (but still storing its value), applying pica resize and then rotating image if needed.
Sorry, as I mentioned I may be deadly wrong here.
- https://github.com/nodeca/nodeca.users/blob/filterjpeg2/lib/image_blob_reduce/index.js#L13-L15
- https://github.com/nodeca/nodeca.users/blob/filterjpeg2/lib/image_blob_reduce/lib/jpeg_plugins.js
Code for patching orientation of input blob is commented out, because not ready. It will be alive soon. But my question is about public API https://github.com/nodeca/nodeca.users/tree/filterjpeg2/lib/image_blob_reduce#api. Is it good or something missed?
@puzrin - yep, missed that.
Regarding API itself - I'm not the best match to describe because I already solved issues manually but from what I see that could be really useful for someone else who faces same problem I had.
https://github.com/nodeca/image-blob-reduce - released.