Equivalent of Blender "auto depth"
thmxv opened this issue · comments
This would require the use of ray casting. But allow much more instinctive navigation in the scene.
The idea is to cast a ray into the scene from the camera position into the direction of the mouse pointer. If there is a hit, use the location of the hit:
- As the rotation center while rotating
- As the "target" while zooming or panning (affecting the radius)
With this enabled, the geometry under the cursor is always the same after the zoom/rotation/pan as it was before.
Hi, thanks for the suggestion, it's a good idea.
As the "target" while zooming or panning (affecting the radius)
The zoom part of this would be "zoom to mouse cursor" in Blender, and you've already created #63 for that. I'll keep these separate like Blender does.
Thanks,
I think, in Blender, the "auto depth" option affects the zoom also. But differently than the "zoom to mouse cursor".
The "zoom to mouse cursor" affects only the direction in which the zoom is done. While the "auto depth" option affect the radius/target depth/sensitivity. But the last one is a bit difficult to tell given that in Blender you can zoom "past" the target once you get too close to it.
I think I know what you mean - when panning, the pivot point is where you clicked, thus changing the distance between the pivot and the camera. That is different from the 'focus' though, because zooming always zooms in and out from the focus, and the focus stays consistently in the middle of the screen.
To be honest, I'm having a lot of trouble implementing this. The main problem I'm having is that I can't just rotate around the raycast hit location, I need to translate that movement back into alpha, beta, and focus. In other words, what is the change in focus, alpha, and beta that would result in the movement of rotating around a different point in space.
As opposed to if the only state was the camera's current transform and the focus, then all movement would simply be a modification of that transform and it would be relatively easy (but cause other problems).
Any help would be appreciated!
I am available to help. However I am new to both rust (coming from C++) and bevy and I tried to study the code of this add-on and still could not understand how it works yet. So this will probably take some time. I am still working on that, but in the meantime, feel free to indicate me on how I can help.
Well the part I'm stuck on has nothing to do with rust, the plugin, or bevy haha.
The overview of how it works is this. There are 4 values that are used to determine the transform of the camera each frame: alpha (angle around global y axis), beta (angle around local x axis), focal point (vec3) and radius (distance from focal point).
On every tick an entirely new transform is created from these values and applied to the camera. Imagine it like this: first it's rotated around global y axis based on alpha (yaw). Then it's rotated around its local x axis based on beta (tilt). Then it's moved/translated to the focal point, and finally it's moved backwards (local Z) the radius amount.
This makes it orbit around the focal point, which is always exactly where the camera is looking at. This is the 'turn table' style orbiting, where 'up' never changes.
Now the hard part comes in two parts:
- Creating the same turn table orbit movement temporarily around a different focal/pivot point
- Translating that movement back into those 4 values - alpha, beta, focus, radius
Radius should actually stay the same, which helps, because once step 1 is done, you can use 'forward * radius' in order to calculate the new focus. And I believe alpha and beta should also stay the same, relative to the new focus.
However I'm actually stuck on step 1 right now. The alpha/yaw movement is easy, but getting the beta/tilt working around an offset pivot point is proving difficult. Using the transform's 'rotate around' doesn't work as intended.
Hopefully that helps you or maybe someone else reading this. Good luck, and no pressure, I'll keep trying myself.
I am still reading the code, but at utils.rs line 35 in the multiplication, both quaternions are after the original quaternion. However in linear algebra, multiplication is not commutative so both transformations are in the local space. I don't think it is related to this issue or even a bug, but it is contradictory to the code in the cheat book and to the description in the previous comment.
I think this line:
new_transform.rotation *= Quat::from_rotation_y(alpha) * Quat::from_rotation_x(-beta);should be changed to this:
new_transform.rotation = Quat::from_rotation_y(alpha) * new_transform.rotation * Quat::from_rotation_x(-beta);Edit: Sorry about that. Given that new_transform is IDENTITY, there is no difference. The * in the *= can also be removed I suppose.
So I did experiment messing with the code and for me rotate_around does work. I did not do the ray cast to detect the pivot_point but hard coded it to be a corner of the cube in the basic example. I changed the file util.rs line 35 to multiple lines:
// new_transform.rotation *= Quat::from_rotation_y(alpha) * Quat::from_rotation_x(-beta);
let rotation = Quat::from_rotation_y(alpha) * Quat::from_rotation_x(-beta);
let pivot_point = Vec3::new(0.5, 1.0, 0.5);
new_transform.rotate_around(pivot_point, rotation);And it works, the corner of the cube in question stays at the same position in the viewport.
Edit: Sorry about that. Given that new_transform is IDENTITY, there is no difference. The * in the *= can also be removed I suppose.
Edit: Sorry about that. Given that new_transform is IDENTITY, there is no difference. The * in the *= can also be removed I suppose.
Yes you're right the *= is unnecessary. Also note that an euler rotation XYZ also works the same:
new_transform.rotation = Quat::from_euler(EulerRot::YXZ, alpha, -beta, 0.0);So I did experiment messing with the code and for me
rotate_arounddoes work. I did not do the ray cast to detect thepivot_pointbut hard coded it to be a corner of the cube in the basic example. I changed the fileutil.rsline 35 to multiple lines:// new_transform.rotation *= Quat::from_rotation_y(alpha) * Quat::from_rotation_x(-beta); let rotation = Quat::from_rotation_y(alpha) * Quat::from_rotation_x(-beta); let pivot_point = Vec3::new(0.5, 1.0, 0.5); new_transform.rotate_around(pivot_point, rotation);And it works, the corner of the cube in question stays at the same position in the viewport.
Thanks! That helps, but it's not the whole solution. When the pivot changes, the camera jumps. You can see that the pivot is always in the vertical center of the screen. This is unlike Blender where the pivot stays in the position on the screen that you clicked.
Screen.Recording.2024-03-11.at.11.37.23.mov
You can find the code used for the above video on this branch: https://github.com/Plonq/bevy_panorbit_camera/tree/64-auto-depth. I'm using the basic example for testing, with smoothing disabled for simplicity. The orange-red gizmo is the pivot and the aquamarine gizmo is the focus.
Also note I made one small change:
new_transform.rotate_around(pivot - focus, rotation);Subtracting focus here means it still works when the focus has changed (panned the camera).
Thanks! That helps, but it's not the whole solution. When the pivot changes, the camera jumps. You can see that the pivot is always in the vertical center of the screen. This is unlike Blender where the pivot stays in the position on the screen that you clicked.
Yes. I figured that since, in utils.rs update_orbit_transform, the alpha and beta parameters represent the whole rotation since the start of the app, using the pivot point there is making the assumption that all the rotations ever were made around the same pivot point. So I started trying as you recommended yesterday and making a "temporary movement" (in lib.rs) to translate into the other properties, however I had no success doing that.
Also, if doing things there: the limits might cause issues. But given that there is not limit for the pan operation, this should be OK.
In my opinion this is probably the best option. And not only radius stays unchanged but alpha and beta also stay unchanged the "only" thing to calculate is a translation that compensate for the rotation around the pivot point.
new_transform.rotate_around(pivot - focus, rotation);
Yes I figured that also. However, without this the rotation start to behave weirdly after a pan operation, but with this the pan operation behave weirdly after a rotate operation.
In my opinion this is probably the best option. And not only radius stays unchanged but alpha and beta also stay unchanged
I'm not so sure. alpha stays the same but beta changes. At least, the beta relative to the pivot point is different to the beta relative to the focus (where the camera is looking). Hopefully this illustration explains:
the "only" thing to calculate is a translation that compensate for the rotation around the pivot point.
Yeah I haven't been successful at this.
Actually, I maybe be wrong with that illustration. If beta is always consistent with the focus and the pivot point rotation is only temporary, then you're right I think beta does stay the same - it's the focus that will move. This hurts my brain 😆
the "only" thing to calculate is a translation that compensate for the rotation around the pivot point.
If we factor in this offset then I believe beta would stay the same 🤔
Yes, I am pretty sure beta/pitch stays the same.
I am making progress, but my linear algebra courses are a long way back. Basically I need to transform a local quaternion into a global one, in order to be able to commute them. I have:
q1 * q2 = q3
where I have q1 and q2 and can calculate q3, but I need to calculate q4 so that:
q4 * q1 = q3
Maybe asking help by someone with a more mathematical mind/expertise for this would be better.
Edit: Sorry I finally managed. Other message coming
Also the limits causes issues, but it can be worked around.
Ok, so I managed to get something working. I:
- used your branch and reverted the changes made to
utils.rsupdate_orbit_transform - commented the apply constrains part in
lib.rspan_orbit_camera - In
lib.rspan_orbit_cameraI added:
let mut transform_temp = transform.clone();
let yaw = Quat::from_rotation_y(-delta_x);
let pitch = Quat::from_rotation_x(-delta_y);
let pivot = *pivot_point;
transform_temp.rotate_around(pivot, yaw);
let pitch_global = transform_temp.rotation * pitch * transform_temp.rotation.inverse();
transform_temp.rotate_around(pivot, pitch_global);
pan_orbit.target_focus = transform_temp.translation
+ (transform_temp.forward() * pan_orbit.target_radius);between
pan_orbit.target_alpha -= delta_x;
pan_orbit.target_beta += delta_y;and
has_moved = true;So after a bit of cleanup, this works also and is a bit quicker:
let mut transform_temp = transform.clone();
let yaw = Quat::from_rotation_y(-delta_x);
let pitch = Quat::from_rotation_x(-delta_y);
let pitch_global =
transform_temp.rotation * pitch * transform_temp.rotation.inverse();
transform_temp.rotate_around(*pivot_point, yaw * pitch_global);
pan_orbit.target_focus = transform_temp.translation
+ (transform_temp.forward() * pan_orbit.target_radius);Still todo:
- If no hit on the raycast, chose a pivot point under the cursor and on the plane perpendicular the the camera forward direction, passing by the focus point.
- Get the
target_alpha/betabefore applyingdelta_x/y, applydelta_x/y, apply the constrains directly after that and recalculate adelta_x/ybased on the values oftarget_alpha/betabefore incrementing them and after applying the constrains. - Auto depth for pan
- Auto depth for zoom
Also in the todo list possibly:
- Change the radius so that the focus is in the same plane (perpendicular to the camera forward direction) as the pivot
Nice work! I implemented the limit fix you mention and that works (no longer pans/slides when you hit the limit).
Unfortunately, there are problems with smoothing. Your solution as-is doesn't work at all with smoothing. Here's what it looks like with default smoothing:
Screen.Recording.2024-03-12.at.18.33.05.mov
However, if transform_temp is a representation of the target values, like so:
let mut transform_temp = Transform::IDENTITY;
transform_temp.rotation = Quat::from_rotation_y(pan_orbit.target_alpha)
* Quat::from_rotation_x(-pan_orbit.target_beta);
transform_temp.translation +=
pan_orbit.target_focus + transform_temp.back() * pan_orbit.target_radius;
let yaw = Quat::from_rotation_y(-delta_x);
let pitch = Quat::from_rotation_x(-delta_y);
let pitch_global = transform_temp.rotation * pitch * transform_temp.rotation.inverse();
transform_temp.rotate_around(*pivot_point, yaw * pitch_global);
pan_orbit.target_focus =
transform_temp.translation + (transform_temp.forward() * pan_orbit.target_radius);Then it sort of works:
Screen.Recording.2024-03-12.at.18.34.36.mov
You can see that it's wobbly. If we change the smoothing values to exaggerate the effect, you can see it very obviously.
Here's with orbit smoothing at 0.5 and pan smoothing disabled.
Screen.Recording.2024-03-12.at.18.36.15.mov
Here's with orbit smoothing disabled and pan smoothing at 0.5.
Screen.Recording.2024-03-12.at.18.36.50.mov
And this made me think maybe if both pan and orbit smoothing are at the same value, then it would 'even out' and be fine, but unfortunately no. It's still off:
Screen.Recording.2024-03-12.at.18.38.15.mov
I believe the problem is that the pan (focus) movement is linear (straight line from old focus to new focus), however the orbit movement is circular, and because the two are combined, then they don't line up and it result in the wobbliness. So if we can make the focus/pan smoothing to follow the same circular movement as the orbit, then they would line up and everything would work. But I have no idea how to go about that.
I had a go at doing this, and the closest I came was to apply the same solution to the actual focus (not the target focus), after interpolating. This code comes after new_focus is declared.
let mut transform_temp = Transform::IDENTITY;
transform_temp.rotation =
Quat::from_rotation_y(alpha) * Quat::from_rotation_x(-beta);
transform_temp.translation += pan_orbit.focus + transform_temp.back() * radius;
let delta_alpha = new_alpha - alpha;
let delta_beta = new_beta - beta;
let yaw = Quat::from_rotation_y(delta_alpha);
let pitch = Quat::from_rotation_x(-delta_beta);
let pitch_global =
transform_temp.rotation * pitch * transform_temp.rotation.inverse();
transform_temp.rotate_around(*pivot_point, yaw * pitch_global);
new_focus = transform_temp.translation + (transform_temp.forward() * radius);This results almost perfect smooth movement, however as you can see in this video, focus ends up drifting away from the target_focus (floating point errors?).
Screen.Recording.2024-03-12.at.20.03.47.mov
We need to somehow interpolate between the current and target focus following the exact same path the target focus took. How to do that accurately, I'm not sure yet.
I've updated my branch with this attempt, along with two commented-out failed attempts at different solutions.
Oh I also implemented the limit fix you described, which works well.
Nice work and progress. Thanks for the feedback, it helps a lot.
It does look like a accumulation error of numerical precision indeed. Hopefully we can fix that by normalizing some (all) the quaternions somewhere (everywhere). I will try to experiment with this and come with a solution but, even though it would hurt to come so close without success, I cannot promise anything. :(
Anyway, by this point we did all the hard work. I put the culpability of this on the smoothing.
Not a numerical precision accumulation error. However I manged to fix the "drift" by switching some values form target_focus/radius to just focus/radius
diff --git a/src/lib.rs b/src/lib.rs
index 1d5dfd0..4364058 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -630,16 +630,20 @@ fn pan_orbit_camera(
let mut transform_temp = Transform::IDENTITY;
transform_temp.rotation = Quat::from_rotation_y(pan_orbit.target_alpha)
* Quat::from_rotation_x(-pan_orbit.target_beta);
+ // transform_temp.translation +=
+ // pan_orbit.target_focus + transform_temp.back() * pan_orbit.target_radius;
transform_temp.translation +=
- pan_orbit.target_focus + transform_temp.back() * pan_orbit.target_radius;
+ pan_orbit.focus + transform_temp.back() * radius;
let delta_alpha = pan_orbit.target_alpha - pre_target_alpha;
let delta_beta = pan_orbit.target_beta - pre_target_beta;
let yaw = Quat::from_rotation_y(delta_alpha);
let pitch = Quat::from_rotation_x(-delta_beta);
let pitch_global = transform_temp.rotation * pitch * transform_temp.rotation.inverse();
transform_temp.rotate_around(*pivot_point, yaw * pitch_global);
+ // pan_orbit.target_focus =
+ // transform_temp.translation + (transform_temp.forward() * pan_orbit.target_radius);
pan_orbit.target_focus =
- transform_temp.translation + (transform_temp.forward() * pan_orbit.target_radius);
+ transform_temp.translation + (transform_temp.forward() * radius);
gizmos.sphere(
pan_orbit.target_focus,However the new version of you code breaks the panning, and I did not managed to restore it in a way that works.
Also doing twice the calculation to rotate around a point is not really optimal and duplicate some code (some of which is also duplicated in util.rs) but the code duplication can be cleaned up when we manage to get something that works.
Making progress! If we can't solve this in a way that work well though then that's okay.
FYI only changing target_focus to focus is necessary from your diff.
So one workaround for the issue of panning breaking, is that we make panning and orbiting mutually exclusive. If panning you can't orbit and if orbiting you can't pan, and doing either interrupts the other. It's not a perfect solution, but it is pretty good. Here's a demo:
Screen.Recording.2024-03-13.at.18.37.49.mov
Anyway, by this point we did all the hard work. I put the culpability of this on the smoothing.
You're right about this. Blender has no smoothing! It definitely complicates everything. If we can't get it working well with smoothing, we could just avoid the problem entirely and say that if this option is enabled, it also disables all smoothing. That would at least be better than nothing for the people that want this feature.
As usual, I've updated the branch with a POC of the above.
Also doing twice the calculation to rotate around a point is not really optimal and duplicate some code (some of which is also duplicated in util.rs) but the code duplication can be cleaned up when we manage to get something that works.
This will be an optional feature, so I'm not fussed about the extra calculation (doubt it's significant), and yeah the code can be cleaned up and better structured if we find a solution
Anyway, by this point we did all the hard work. I put the culpability of this on the smoothing.
You're right about this. Blender has no smoothing! It definitely complicates everything. If we can't get it working well with smoothing, we could just avoid the problem entirely and say that if this option is enabled, it also disables all smoothing. That would at least be better than nothing for the people that want this feature.
I think it will be possible, thanks for your cooperation on this. I would definitely not have managed to do that alone.
I modified the code to have a pivot point under the mouse even when there is no hit during the ray cast:
diff --git a/src/lib.rs b/src/lib.rs
index 4185b3c..582e527 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -419,7 +419,11 @@ fn pan_orbit_camera(
let hits1 = raycast.cast_ray(cursor_ray, &default());
if let Some(hit) = hits1.first().map(|(_, hit)| hit) {
*pivot_point = hit.position();
- *pivot_radius = pivot_point.distance(pan_orbit.target_focus);
+ // *pivot_radius = pivot_point.distance(pan_orbit.target_focus);
+ } else {
+ let factor = transform.forward().dot(cursor_ray.direction.into());
+ *pivot_point = cursor_ray.origin
+ + cursor_ray.direction * (pan_orbit.radius.unwrap() / factor);
}
}
}That one was easy.
Edit: No so easy after all. I modified the diff to include the dot product factor so that they are truly in the same plane, not close to it.
I suppose now I will focus on the pan and zoom. Or maybe to move the focus to be in the same plane as the pivot point during orbit. If I understood well the attempts in you code this is something needed also?
Edit: A fix to do also: the values do not always converge to be exactly equal to the target values. But very close to it due to precision errors during the calculation. It looks right but does the update calculation when not needed.
So I decided to concentrate on moving the focus to be on the same plane (perpendicular to the camera forward direction) as the pivot point. After all this is pretty much the only things this "auto depth" option have in common between orbit pan and zoom (even when "zoom to mouse position" is not set) in Blender.
Doing the math/geometry was easy but doing the code so that the smoothing was still working and there was no jumping at the beginning or end of the orbit operation was a real pain. Here is a video of the feature working:
Screencast.from.2024-03-13.17-29-46.webm
Here a diff file with the changes for this, the previous pivot_point calculation when no hit by the raycast and some basic changes: Like do not use the same value for pan/orbit smoothing anymore and set the pan smooth value to something non zero in the demo.
So I just taught of this, but my last changes can break the radius limit. Also the whole things since the start is probably not working with orthographic cameras. :(
I think it will be possible, thanks for your cooperation on this. I would definitely not have managed to do that alone.
I like the optimism! I don't think I would have figured out the math myself (or given up first), so we're a good team!
So I just taught of this, but my last changes can break the radius limit. Also the whole things since the start is probably not working with orthographic cameras. :(
Yeah there is no obvious solution here. We have two choices:
- Prevent the focus (and pivot point?) moving further away from the camera than the max radius limit; or
- Snap that camera to the max radius if a pivot point is chosen farther away
I couldn't even get option 1 working, but here's option 2. Not very nice.
Screen.Recording.2024-03-14.at.18.21.53.mov
Side note: I moved the bottom if is_orbiting block after util::update_orbit_transform so that zooming still works while orbiting.
I figured out option 1. It's much better than option 2. Awkward, but acceptable outcome if you choose to add zoom limit.
Screen.Recording.2024-03-14.at.18.36.23.mov
As usual branch updated.
Nice,
On my side I fixed some trivial stuff:
- Fixed the calculation of the pivot point when there is no ray cast hit. It turns out that
cursor_ray.originis not the same as the camera position which introduced a small error (not noticeable visually)
I did some small cleanups:
- Removed useless comment regarding the pan_smoothness begin the same as the orbit_smoothness which is not true and the code is not there anymore
- Removed a useless
+from a+=
I "think" I also fixed the orthographic projection related stuff:
- For this I switched to the orthographic example but I modified the
nearandfarparameters. Otherwise thenearis by default set to 0.0 and this makes sometimes some bad math appear correct - I calculated the pivot point when using an orthographic projection for the camera an there is no hit by the ray cast
- When modifying the radius to move the focus point on the same plane as the pivot. We do not want this change in radius to affect the projection scale. I did not found a way to do that cleanly, so I did an ugly hack for this but it "works" and is fairly simple/understandable.
Here is a diff of all my changes related to the above points.
Good work on the orthographic, though I don't particularly like the hack. I used to have a separate scale field on PanOrbitCamera which was used for orthographic projection, but this was recently removed in #58. Your hack seems to be essentially the same thing - separating out ortho scale from radius - just as a factor rather than entirely separate. So I wonder whether it would be better to revert that PR? What do you think?
Another thing - I get stutter/jitter in ortho. This is due to the 1-frame delay due to changing values after update_orbit_transform has already run (meaning those changes only get applied the next frame). A hacky/naive/brute force solution is to run update_orbit_transform twice - before and after the if is_orbiting block. Really not a fan of that though, and I'm sure there's a way to avoid that by incorporating zoom into the is_orbiting block instead, then only calling update_orbit_transform afterwards.
I'll look into this when I can.
Demo of stutter:
Screen.Recording.2024-03-16.at.12.07.40.mov
Good work on the orthographic, though I don't particularly like the hack. I used to have a separate scale field on PanOrbitCamera which was used for orthographic projection, but this was recently removed in #58. Your hack seems to be essentially the same thing - separating out ortho scale from radius - just as a factor rather than entirely separate. So I wonder whether it would be better to revert that PR? What do you think?
I do not like the hack neither. In some ways the scale field, has before, was better but in other ways it makes it more difficult to switch from perspective to orthographic projections and have a relatively identical field of view. I do not really have a strong opinion on that one. None of the "solutions" are really clean. I suppose my hack of having a scale factor defaulted to one and not needing any work from the user to calculate a correct value to set the scale is easier.
Another thing - I get stutter/jitter in ortho. This is due to the 1-frame delay due to changing values after update_orbit_transform has already run (meaning those changes only get applied the next frame). A hacky/naive/brute force solution is to run update_orbit_transform twice - before and after the if is_orbiting block. Really not a fan of that though, and I'm sure there's a way to avoid that by incorporating zoom into the is_orbiting block instead, then only calling update_orbit_transform afterwards.
I cannot reproduce the pivot point jitter here.l
Edit: Actually I can reproduce the jitter. I forgot to undo/comment some of the changes that actually moved the update_orbit_transform after the code that moves the focus point.
But yes: The is_orbiting block that changes the focus point to be at the same depth as the pivot can be moved before applying the values with update_orbit_transform. The only thing it does differently is allowing zooming while keeping the orbit mouse button pressed, but not while while actually orbiting (moving the mouse in addition to pressing the orbit button).
In addition in the future (when also supporting auto_depth for panning and zooming) this code will need to be always called (when auto_depth is set to true at least).
I am currently working on this and managed to get the focus point relocated correctly by:
- Changing the condition to calculate
pivot_pointfrom
if input::orbit_just_pressed(&pan_orbit, &mouse_input, &key_input)
{
...to
if input::orbit_just_pressed(&pan_orbit, &mouse_input, &key_input)
|| input::pan_just_pressed(&pan_orbit, &mouse_input, &key_input)
|| mouse_key_tracker.scroll_line != 0.0
{
...- Removing this
if is_orbitingcondition and always execute the code to move the focus point at the end (the one talked about a bit higher in my message)
However, this moves the focus point to the correct position but breaks the zooming and panning which do not work anymore and I do not manage to fix this :(
Regarding the hack. There might be a way to avoid it just by setting the radius to a different value.
I managed to avoid it by replacing the line:
pan_orbit.ortho_scale_factor *= pan_orbit.radius.unwrap() / new_radius;by
if let Projection::Orthographic(_) = *projection {
new_radius = radius;
}And thus not using the hack and allowing to remove the ortho_scale_factor from everywhere.
I like the avoidance of the hack, but it does break zoom while orbiting because it's using radius which is the old value not the new value. Unfortunately, using the new value (before modifications) results the same camera jump when the pivot changes :(
Look, if I'm honest, I'm starting to feel this is getting to be too complicated. Even if we solve the issues, it's still a whole lot of complexity stemming from making it work with smoothing, which makes bugs more likely, and also makes it harder to understand and harder to add other features as well.
I think it would probably be better to create a separate plugin that doesn't have smoothing, and implement this there.
I'm still open to implementing this, but it just seems there are a lot of hacks and special cases and I am not a fan.
I understand your thinking. I understand the geometry and linear algebra of it but I have difficulties to understand some things related to the smoothing too. Like why the new focus is moved at the end of the code when it should probably moved at the start, and why some calculation need to be done twice for the smoothing interpolation to be working. Sorry to have wasted your time so much and thank you a lot for the help.
You haven't wasted my time! I'm glad we explored this, and thank you for the help. I think it's just not going to fit well with the architecture of this plugin. Particularly the fact the plugin uses absolute positioning and smoothes by tracking two separate values. A camera controller that uses relative positions and velocity+acceleration for smoothing I think would be much more suitable. In fact, bevy_editor_cam might be that plugin as it does use relative positioning. Perhaps you could request the feature there? (Based on the docs, I even think it already does panning based on raycast hit location which is part of the auto-depth feature).