RenderTarget with h2d.Scene doesn't scale properly
hazzen opened this issue · comments
I'm attempting to use Heap's render-to-texture support to pixelate a scene. I do this by using a fixed-size scene, rendering it to RenderTarget, and using that texture as a h2d.Bitmap in a 2d scene. As I want some non-pixellated UI features, the basic code looks like:
class PixellatedApp extends hxd.App {
final WIDTH = 640;
final HEIGHT = 480;
var renderTargetBitmap:h2d.Bitmap;
var renderTarget:h3d.mat.Texture;
var renderScene:h2d.Scene;
override function init() {
renderScene = new h2d.Scene();
renderScene.scaleMode = Fixed(width, height, 1);
renderTarget = new h3d.mat.Texture(width, height, [Target]);
renderTargetBitmap = new h2d.Bitmap(h2d.Tile.fromTexture(renderTarget), s2d);
// ...use renderScene for stuff ...
}
override function render(e:h3d.Engine) {
// Scale up the target to fill the screen.
renderTargetBitmap.scaleX = engine.width / width;
renderTargetBitmap.scaleY = engine.height / height;
// Render to the target.
e.pushTarget(renderTarget);
e.clear(0xff660000, 1);
renderScene.render(e);
e.popTarget();
super.render(e);
}
}However, this does not produce expected results. There are two separate bugs; one I can explain and one I can't. The first bug is easy. When using a Fixed scaling mode, Scene.calcViewport assumes it is rendering to a window-sized surface (via engine.width and engine.height):
Lines 349 to 379 in 5e07799
I think this code should be using some other measure of width/height, perhaps derived from a render target if present? It looks more complicated than I'm capable of suggesting a fix for, unfortunately, as the code is only called on resize events and it would instead need to be called every frame.
I can work around the first bug by scaling/offsetting renderScene by window-based amounts:
renderScene.scaleX = e.width / width;;
renderScene.scaleY = e.height / height;;
renderScene.x = (width - e.width) / 2;
renderScene.y = (height - e.height) / 2;The second bug is more confusing. Namely, the scale used by h2d.Scene seems to be a frame behind. If you resize the window by a large amount (maximize, for instance), the scaling issue will return. Working around this one is done in much the same way as the first fix, but requires tracking the current and the previous window sizes. I have a sample app to demonstrate all of this, plus it can use a 3d scene rendered to a texture to demonstrate the lack of issue there.
As I have a sufficient workaround, I'm not expecting a speedy response/fix, but any effort would be appreciated (again, I'd send a fix myself but it's more complicated than I feel comfortable fixing in a codebase I've never contributed to). You can find the sample app inline:
class ToggleText {
public var val(default, set):Bool;
final label:String;
final text:h2d.Text;
public function new(label:String, parent:h2d.Object) {
this.label = label;
text = new h2d.Text(hxd.res.DefaultFont.get(), parent);
text.text = '${label}: ${val}';
}
public function set_val(val:Bool) {
this.val = val;
text.text = '${label}: ${val}';
return val;
}
public function toggle() {
this.val = !val;
}
}
class ScalingTest extends hxd.App {
var width:Int;
var height:Int;
var windowWidth:Int;
var windowHeight:Int;
var previousWidth:Int;
var previousHeight:Int;
var renderTargetBitmap:h2d.Bitmap;
var renderTarget:h3d.mat.Texture;
var renderScene:h2d.Scene;
var renderScene3d:h3d.scene.Scene;
var do3d:ToggleText;
var doScale:ToggleText;
var doFix2d:ToggleText;
var doResizeAdjust:ToggleText;
override function init() {
super.init();
width = Std.int(engine.width / 4);
height = Std.int(engine.height / 2);
windowWidth = engine.width;
windowHeight = engine.height;
previousWidth = engine.width;
previousHeight = engine.height;
engine.backgroundColor = 0x222222;
renderScene = new h2d.Scene();
renderScene.scaleMode = Fixed(width, height, 1);
var gfx = new h2d.Graphics(renderScene);
gfx.lineStyle(Math.min(width, height) / 10, 0xffffff);
gfx.moveTo(0, 0);
gfx.lineTo(width, height);
gfx.moveTo(0, height);
gfx.lineTo(width, 0);
renderScene3d = new h3d.scene.Scene();
new h3d.scene.fwd.DirLight(new h3d.Vector(0.5, 0.5, -0.5), renderScene3d);
var cube = new h3d.prim.Cube();
cube.addNormals();
cube.translate(-0.5, -0.5, -0.5);
var mesh = new h3d.scene.Mesh(cube, renderScene3d);
mesh.material.color.setColor(0xffea8220);
renderTarget = new h3d.mat.Texture(width, height, [Target]);
renderTarget.depthBuffer = new h3d.mat.DepthBuffer(width, height);
renderTargetBitmap = new h2d.Bitmap(h2d.Tile.fromTexture(renderTarget), s2d);
var flow = new h2d.Flow(s2d);
flow.horizontalSpacing = 10;
do3d = new ToggleText('[3]: do3d', flow);
doScale = new ToggleText('[S]: doScale', flow);
doFix2d = new ToggleText('[F]: doFix2d', flow);
doResizeAdjust = new ToggleText('[R]: doResizeAdjust', flow);
}
override function onResize() {
previousWidth = windowWidth;
previousHeight = windowHeight;
windowWidth = engine.width;
windowHeight = engine.height;
}
override function update(dt:Float) {
if (hxd.Key.isPressed(hxd.Key.NUMBER_3)) {
do3d.toggle();
}
if (hxd.Key.isPressed(hxd.Key.S)) {
doScale.toggle();
}
if (hxd.Key.isPressed(hxd.Key.F)) {
doFix2d.toggle();
}
if (hxd.Key.isPressed(hxd.Key.R)) {
doResizeAdjust.toggle();
}
}
override function render(e:h3d.Engine) {
e.pushTarget(renderTarget);
e.clear(0xff660000, 1);
// Scale the rendered image to fit the screen.
if (doScale.val) {
renderTargetBitmap.scaleX = engine.width / width;
renderTargetBitmap.scaleY = engine.height / height;
} else {
renderTargetBitmap.scaleX = 1;
renderTargetBitmap.scaleY = 1;
}
if (do3d.val) {
renderScene3d.render(e);
} else {
// Fix the "uses engine size to scale" by re-scaling the scene to match
// the render texture.
if (doFix2d.val) {
renderScene.scaleX = e.width / width;
renderScene.scaleY = e.height / height;
renderScene.x = (width - e.width) / 2;
renderScene.y = (height - e.height) / 2;
} else {
renderScene.scaleX = 1;
renderScene.scaleY = 1;
renderScene.x = 0;
renderScene.y = 0;
}
// The weird one - the resize seems to be behind by a frame, so add an
// additional amount based on the difference.
if (doResizeAdjust.val) {
renderScene.scaleX *= previousWidth / windowWidth;
renderScene.scaleY *= previousHeight / windowHeight;
renderScene.x += (windowWidth - previousWidth) / 2;
renderScene.y += (windowHeight - previousHeight) / 2;
}
renderScene.render(e);
}
e.popTarget();
super.render(e);
}
}. ..and I've just realized my bug. The RenderTarget is already the appropriate size, so my render scene should use Stretch and not Fixed scale mode. Will confirm this solves all my issues when I'm next at my computer.
I'll show myself out - using renderScene.scaleMode = Stretch(width, height); solves the problem. Thinking about this more, this explains why I saw no issues in the 3d scene, as that renderer is effectively always using a full-screen viewport. As I already had a WIDTHxHEIGHT target texture, I don't need to further make my render viewport WIDTHxHEIGHT pixels. The proper incantation to do what I'm trying is:
class PixellatedApp extends hxd.App {
final WIDTH = 640;
final HEIGHT = 480;
var renderTargetBitmap:h2d.Bitmap;
var renderTarget:h3d.mat.Texture;
var renderScene:h2d.Scene;
override function init() {
s2d.scaleMode = /* anything you want */;
renderScene = new h2d.Scene();
renderScene.scaleMode = Stretch(width, height);
renderTarget = new h3d.mat.Texture(width, height, [Target]);
renderTargetBitmap = new h2d.Bitmap(h2d.Tile.fromTexture(renderTarget), s2d);
// ...use renderScene for stuff ...
}
override function render(e:h3d.Engine) {
// Scale up the target to fill the screen. It's important to use s2d's measurements
// here, to allow for any scaleMode it might be using.
renderTargetBitmap.scaleX = s2d.width / width;
renderTargetBitmap.scaleY = s2d.height / height;
// Render to the target.
e.pushTarget(renderTarget);
e.clear(0xff660000, 1);
renderScene.render(e);
e.popTarget();
super.render(e);
}
}