Can't zoom after switching between cameras

Question

Can't zoom after switching between cameras

vinkovsky opened this issue 4 months ago · comments

Arkady Vinkovsky commented 4 months ago

Describe the bug

After dynamically updating the camera from perspective to orthographic, zooming in orthographic mode will not work.

To Reproduce

Steps to reproduce the behavior:

Go to this demo
Click on the top edge of the view cube or rotate the camera to position it above the scene cubes.
Your camera now switches to ortho mode.
Try zooming in. You will not make it
Go to camera-view.tsx to see the code

Code

class Controls {
  perspectiveCamera: PerspectiveCameraImpl;
  orthographicCamera: OrthographicCameraImpl;
  currentCamera: PerspectiveCameraImpl | OrthographicCameraImpl;
  object: CameraControlsImpl;

  constructor(private container: HTMLElement, private scene: Scene) {
    this.perspectiveCamera = new PerspectiveCameraImpl(
      70,
      this.container.clientWidth / this.container.clientHeight,
      0.1,
      4000
    );
    this.orthographicCamera = new OrthographicCameraImpl(
      this.container.clientWidth / -2,
      this.container.clientWidth / 2,
      this.container.clientHeight / 2,
      this.container.clientHeight / -2,
      0.1,
      4000
    );
    this.currentCamera = this.perspectiveCamera;

    const subsetOfTHREE = {
      Vector2: Vector2,
      Vector3: Vector3,
      Vector4: Vector4,
      Quaternion: Quaternion,
      Matrix4: Matrix4,
      Spherical: Spherical,
      Box3: Box3,
      Sphere: Sphere,
      Raycaster: Raycaster,
    };

    CameraControlsImpl.install({ THREE: subsetOfTHREE });

    this.object = new CameraControlsImpl(this.currentCamera, this.container);
    this.object.setPosition(1, 1, 1);
    this.object.setTarget(0, 0, 0);
    this.object.restThreshold = 0.1;
    this.object.dollyToCursor = true;
  }

  dispose() {
    this.object.disconnect();
    this.object.dispose();
  }

  update(delta: number) {
    if (this.object.polarAngle <= 0.01) {
      if (this.currentCamera.type === "PerspectiveCamera") {
        this.setOrthographicCamera();
      }
    } else if (this.currentCamera.type === "OrthographicCamera") {
      this.setPerspectiveCamera();
    }

    this.object.update(delta);
  }

  private updateOrthographicCameraFrustum() {
    const { distance } = this.object;

    const halfWidth =
      frustumWidthAtDistance(this.perspectiveCamera, distance) / 2;
    const halfHeight =
      frustumHeightAtDistance(this.perspectiveCamera, distance) / 2;
    const halfSize = { x: halfWidth, y: halfHeight };
    this.orthographicCamera.top = halfSize.y;
    this.orthographicCamera.bottom = -halfSize.y;
    this.orthographicCamera.left = -halfSize.x;
    this.orthographicCamera.right = halfSize.x;
  }

  updateFrustum() {
    this.perspectiveCamera.aspect =
      this.container.clientWidth / this.container.clientHeight;
    this.updateOrthographicCameraFrustum();
    this.currentCamera.updateProjectionMatrix();
  }

  setOrthographicCamera() {
    this.orthographicCamera.position.copy(this.perspectiveCamera.position);
    this.updateOrthographicCameraFrustum();
    this.currentCamera = this.orthographicCamera;
    this.object.camera = this.orthographicCamera;
    this.orthographicCamera.updateProjectionMatrix();
  }

  setPerspectiveCamera() {
    const oldY = this.perspectiveCamera.position.y;
    this.perspectiveCamera.position.copy(this.orthographicCamera.position);
    this.perspectiveCamera.position.y = oldY / this.orthographicCamera.zoom;
    this.currentCamera = this.perspectiveCamera;
    this.object.camera = this.perspectiveCamera;
    this.perspectiveCamera.updateProjectionMatrix();
  }
}

function frustumHeightAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number
) {
  const vFov = (camera.fov * Math.PI) / 180;
  return Math.tan(vFov / 2) * distance * 2;
}

function frustumWidthAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number
) {
  return frustumHeightAtDistance(camera, distance) * camera.aspect;
}

Live example

demo

codesandbox

repo

Expected behavior

Zoom must work in ortho mode

Screenshots or Video

Screen.Recording.2024-04-22.at.00.31.43.mov

Device

Desktop

OS

MacOS

Browser

Chrome

Muralitob commented 4 months ago

same bug

seanhouli · Answer 1 · Sat May 04 2024 03:55:52 GMT+0800 (China Standard Time)

I needed this functionality as well, I took your demo and got it working: https://codesandbox.io/p/github/vinkovsky/switching-cameras-demo/csb-jzt6zp/draft/gallant-cannon?file=%2Fapp%2Fcamera-view.tsx. I commented out the automatic switching and just bound to 't' to toggle between to make debugging easier. its pretty satisfying to use 😄 . I also am using the getPosition() from camera-controls rather than .position as it's my understanding that's more correct.

Arkady Vinkovsky · Answer 2 · Sat May 04 2024 04:51:31 GMT+0800 (China Standard Time)

@seanhouli Hello! Thanks for your reply. Unfortunately couldn't open your solution :(

seanhouli · Answer 3 · Sat May 04 2024 05:15:17 GMT+0800 (China Standard Time)

@seanhouli Hello! Thanks for your reply. Unfortunately couldn't open your solution :(

Can you try again? I have not used codesandbox a ton, but under sharing it does say it's publicly accessible.

Also, upon playing with it more, there does seem to be issues with the swapping code itself giving strange results when you stress test it, probably because camera-controls has its own quirks

edit: sent a friend the same link and could open it just fine

Arkady Vinkovsky · Answer 4 · Sat May 04 2024 05:17:05 GMT+0800 (China Standard Time)

@seanhouli still not working

seanhouli · Answer 5 · Sat May 04 2024 05:24:03 GMT+0800 (China Standard Time)

All I can do is share the link, which I have done 🤷

seanhouli · Answer 6 · Sat May 04 2024 05:26:15 GMT+0800 (China Standard Time)

@vinkovsky Here are the contents of the camera-view.tsx file if you still cannot use the link.

import { useEffect, useMemo } from "react";
import { useFrame, useThree } from "@react-three/fiber";
import {
  PerspectiveCamera as PerspectiveCameraImpl,
  OrthographicCamera as OrthographicCameraImpl,
  Vector3,
  Vector2,
  Vector4,
  Quaternion,
  Matrix4,
  Spherical,
  Box3,
  Raycaster,
  Sphere,
  EventDispatcher,
  Scene,
} from "three";
import CameraControlsImpl from "camera-controls";

export function Camera() {
  const gl = useThree((state) => state.gl);

  const set = useThree((state) => state.set);
  const get = useThree((state) => state.get);
  const scene = useThree((state) => state.scene);

  const controls = useMemo(
    () => new Controls(gl.domElement, scene),
    [gl.domElement, scene],
  );

  useEffect(() => {
    const onkeydown = (e: KeyboardEvent) => {
      console.log(e.key);
      if (e.key.toLowerCase() === "t") {
        controls.currentCamera === controls.perspectiveCamera
          ? controls.setOrthographicCamera()
          : controls.setPerspectiveCamera();
      }
    };

    window.addEventListener("keydown", onkeydown);

    return () => {
      window.removeEventListener("keydown", onkeydown);
    };
  }, [controls]);

  useEffect(() => {
    if (scene.children.length) {
      fitCameraToSceneBoundingSphere(controls.object, scene);
    }

    console.log("scene.children.length", scene.children.length);
  }, [controls.object, scene.children]);

  useFrame((_, delta) => {
    controls.update(delta);
    gl.render(scene, controls.currentCamera);
  }, -1);

  useEffect(() => {
    const oldControls = get().controls;
    const oldCamera = get().camera;
    set({
      controls: controls.object as unknown as EventDispatcher,
      camera: controls.currentCamera,
    });
    return () => set({ controls: oldControls, camera: oldCamera });
  }, [controls, get, set]);

  return (
    <>
      <primitive object={controls.object} />
      <primitive object={controls.currentCamera} />
    </>
  );
}

// https://gist.github.com/nickyvanurk/9ac33a6aff7dd7bd5cd5b8a20d4db0dc
class Controls {
  perspectiveCamera: PerspectiveCameraImpl;
  orthographicCamera: OrthographicCameraImpl;
  currentCamera: PerspectiveCameraImpl | OrthographicCameraImpl;
  object: CameraControlsImpl;

  constructor(
    private container: HTMLElement,
    private scene: Scene,
  ) {
    this.perspectiveCamera = new PerspectiveCameraImpl(
      70,
      this.container.clientWidth / this.container.clientHeight,
      0.1,
      4000,
    );
    this.orthographicCamera = new OrthographicCameraImpl(
      this.container.clientWidth / -2,
      this.container.clientWidth / 2,
      this.container.clientHeight / 2,
      this.container.clientHeight / -2,
      0.1,
      4000,
    );
    this.currentCamera = this.perspectiveCamera;

    const subsetOfTHREE = {
      Vector2: Vector2,
      Vector3: Vector3,
      Vector4: Vector4,
      Quaternion: Quaternion,
      Matrix4: Matrix4,
      Spherical: Spherical,
      Box3: Box3,
      Sphere: Sphere,
      Raycaster: Raycaster,
    };

    CameraControlsImpl.install({ THREE: subsetOfTHREE });

    this.object = new CameraControlsImpl(this.currentCamera, this.container);
    this.object.setPosition(1, 1, 1);
    this.object.setTarget(0, 0, 0);
    this.object.restThreshold = 0.1;
    this.object.dollyToCursor = true;
  }

  dispose() {
    this.object.disconnect();
    this.object.dispose();
  }

  update(delta: number) {
    // if (this.object.polarAngle <= 0.01) {
    //   if (this.currentCamera.type === "PerspectiveCamera") {
    //     this.setOrthographicCamera();
    //   }
    // } else if (this.currentCamera.type === "OrthographicCamera") {
    //   this.setPerspectiveCamera();
    // }

    this.object.update(delta);
  }

  private updateOrthographicCameraFrustum() {
    const { distance } = this.object;

    const halfWidth =
      frustumWidthAtDistance(this.perspectiveCamera, distance) / 2;
    const halfHeight =
      frustumHeightAtDistance(this.perspectiveCamera, distance) / 2;
    const halfSize = { x: halfWidth, y: halfHeight };
    this.orthographicCamera.top = halfSize.y;
    this.orthographicCamera.bottom = -halfSize.y;
    this.orthographicCamera.left = -halfSize.x;
    this.orthographicCamera.right = halfSize.x;
  }

  updateFrustum() {
    this.perspectiveCamera.aspect =
      this.container.clientWidth / this.container.clientHeight;
    this.updateOrthographicCameraFrustum();
    this.currentCamera.updateProjectionMatrix();
  }

  private lastPerspectivePosition: Vector3 = new Vector3();

  setOrthographicCamera() {
    const perspectivePos = this.object.getPosition(new Vector3());
    this.updateOrthographicCameraFrustum();
    this.currentCamera = this.orthographicCamera;
    this.object.camera = this.currentCamera;
    this.object.setPosition(
      perspectivePos.x,
      perspectivePos.y,
      perspectivePos.z,
    );
    this.orthographicCamera.updateProjectionMatrix();

    this.lastPerspectivePosition.copy(perspectivePos);

    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.ZOOM;
  }

  setPerspectiveCamera() {
    const orthoPos = this.object.getPosition(new Vector3());

    const oldY = this.lastPerspectivePosition.y;
    this.currentCamera = this.perspectiveCamera;
    this.perspectiveCamera.updateProjectionMatrix();
    this.object.camera = this.currentCamera;
    this.object.setPosition(
      orthoPos.x,
      oldY / this.orthographicCamera.zoom,
      orthoPos.z,
    );

    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.DOLLY;
  }
}

function frustumHeightAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number,
) {
  const vFov = (camera.fov * Math.PI) / 180;
  return Math.tan(vFov / 2) * distance * 2;
}

function frustumWidthAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number,
) {
  return frustumHeightAtDistance(camera, distance) * camera.aspect;
}

export const fitCameraToSceneBoundingSphere = (
  controls: CameraControlsImpl,
  scene: Scene,
) => {
  const boundingBox = new Box3();
  const center = new Vector3();
  const sphere = new Sphere();

  const defaultRadius = 1;
  const bbox = boundingBox.setFromObject(scene);

  if (controls.fitToSphere) {
    // https://stackoverflow.com/a/63243915/11416728
    controls.fitToSphere(
      bbox.getBoundingSphere(sphere.set(bbox.getCenter(center), defaultRadius)),
      true,
    );
  }
};

seanhouli · Answer 7 · Sat May 04 2024 06:10:04 GMT+0800 (China Standard Time)

@vinkovsky it seems that reverting back to your original version that uses .position is actually working better for me, no idea why. Here are the changes I have made

  setOrthographicCamera() {
    this.object.cancel()
    // .... same as yours
    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.ZOOM
  }

  setPerspectiveCamera() {
    this.object.cancel()
    // .... same as yours
    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.DOLLY
  }

still seeing weirdness when stress-testing hard and swapping back and forth, panning, orbiting aggressively

Arkady Vinkovsky · Answer 8 · Sat May 04 2024 13:58:43 GMT+0800 (China Standard Time)

@seanhouli Thanks for the code snippet! I appreciate it. Yes, I also noticed differences in behavior between .position and .getPosition(). Thats why i used the first one

Muralitob · Answer 9 · Tue May 07 2024 09:58:32 GMT+0800 (China Standard Time)

@seanhouli genius！ it work！