Problems with setPixelRatio since "new render pipeline system" update
vero-so opened this issue · comments
Hi,
I have recently tested with an updated version, and noticed some strange scaling of the viewport (covers on 0.4 of the viewport in each dimension) and traced this change of behavior to the update " feat: new render pipeline system (#1982)" by @christjt . I noticed, that when I comment out setPixelRatio (or use the default 1.0, even though I have and had factor 2.5), the entire viewport is covered -- although it seems that the renderer is not rendering at full possible resolution.
I have run the "simple" example page, where setPixelRatio is also used (with factor 2.5 as well btw), and it works. Do you know what might be causing this? Are some updates needed in our code to make it work again after this update? The update was quite large, so it is a bit hard to find the cause and fix it, so maybe you have a hunch?
Thanks a lot.
Getting called out for introducing bugs 😥 Thank you for testing! Will look at this today. I believe we perhaps have some handling of device pixel ratio when using Cognite3DViewer, which will not be reflected in the "simple" example. May or may not be related, we actually "cap" the maximum resolution of the image for performance reasons. I will look into creating an opt-out option such that you can have full resolution (but we need to have this cap as a default, as low-end devices with high resolution get pretty bad performance).
I'll try to get this fixed. But to be clear, the issue is that setting device pixel ratio mangles the viewport, correct? Having a capped resolution is intended, but will try to create an option to remove this cap.
I got a little time to take a look and I was able to reproduce your findings; both the issue you reported as well as it working fine in "simple" example. Simple also uses the new rendering pipeline system so definitely odd that they are having different behaviours. Will continue to look into this.
Hey, that was rather meant to be a calling for help ;)
Great that you already had some time for looking into this. I am really curious what causes it. Thanks again.
Okey sooo... Finally got to the bottom of this and there is a few things I am not completely sure how we should handle.
Let's take the bug first, this is caused by users setting a devicePixelRatio(DPR) other than 1.0 which under the hood will rescale the actual resolution being rendered. Turn out that renderer.getSize(...); is not actually the resolution being rendered, but rather this multiplied by the devicePixelRatio. So what happens is that we use framebuffers (i.e. render targets) in our render pipeline which is then incorrectly initialized to be the size of the renderer (excluding the DPR). The results of these framebuffers are later read by fetching a specific pixel which is then read wrong since the final output to the canvas has a different resolution than the framebuffer it is reading from. This is why everything seems to be "scaled".
Now... I have a working fix for this, but it will require Reveal to be somewhat opinionated. Currently, Reveal has a hardcoded resolution cap that we enforce by downscaling the renderSize if needed (1.4e6 number of pixels, based on "finger-in-the-air" testing). We don't account for DPR here. so you can effectively bypass this cap by setting the viewer.renderer.setPixelRatio(...);.
Our previous assumptions on the matter is that we simply don't handle DPR at all and assume that no one will set it. What happens in practice then is that devices with a high DPR (f.ex. iPhones) render at a much lower resolution, but the quality will be relatively consistent across devices since the physical size of the virtual pixel will be more or less the same across devices. The downside is of course that you are not able to fully utilize the resolution of the device, but even if you did I am sure that you would see quite a large performance regression.
So this leaves us at a point where we have to either make sure to downscale additionally based on DPR which then ensures the property of a virtual pixel being relatively the same size on any device which yields consistent quality across devices, or take DPR into account when ensuring the resolution cap is met (which then causes the virtual pixel to vary in size across devices). FYI. I am planning to add the ability for the user to control this resolution cap (maybe this is something you guys could contribute?).
Sorry for the wall of text, any thoughts?
@vero-so Please give master a try now. Should fix the bug.
Thanks, gonna try now!
Works now, thanks!