Implement 3d picking of shapes
kaveh808 opened this issue · comments
[Need someone for this.]
Initially by clicking on just shapes in the 3D view.
Requires some knowledge of OpenGL.
Later, it can be extended to mesh parts (face, edge, vertex) of poly-mesh at some point.
I would like to tackle this, I'll start this weekend.
In a modern version of GL, or any of the low overhead APIs, this would be best accomplished using occlusion queries and writing ids as color, but implementing it that way feels pointless without framebuffer support (we would stall the pipeline waiting for the query to finish because we don't have another framebuffer, and so can't do the main render while we wait), so for now I will write it using the select buffer APIs.
Excellent. Do keep in mind that we are currently using OpenGL 3 on Windows/Linux, but OpenGL 2 on MacOS.
Currently the polyhedron class supports arbitrary faces. There is a very naive triangulation method, only guaranteed to work with convex faces.
If krma gets used, your opengl programming will be in vain. There is a "cherno" video on youtube demonstrating how to do this, it may have used vulkan, but it writes ids as color in a separate image and uses the mouse coordinates to extract the object id.
Yes, I do like the object id based solutions as I mentioned. That's more or less what an occlusion query based solution would do as well; the difference is that you can toggle the depth test when you do an occlusion query to count anything in the scissor/viewport as a hit, which means you can support both selecting occluded objects and box selection (you just change the scissor rect to match the box instead of the cursor) almost trivially. Another nice thing with both of these methods over a raymarch is that, since they operate on the results of the pipeline rather than AABBs, they will always be pixel perfect, and all of this without needing to do anything complex in software/compute.
In any case, you may be better served to use vulkan.
I agree that Vulkan (and Metal, since I presume we don't want to rely on MoltenVk for MacOS) would be better for many things, including this. My main concern is that it seems not great to have Vulkan/Metal and GL contexts coexisting in the same application, especially because the current code is using the fixed function pipeline and so would have to duplicate vertex data between three APIs (no GL buffer exposed to share with extensions or anything like that). Even if we did do it in Vulkan right now, the code would have to change drastically to fit within a new backend anyway. To me, this indicates that we should either do it in GL for now and replace it with OQs later, or punt it until the rendering backend has settled.
I completely agree that no GL 3.3 backend should be written, you can dump GL with better APIs on all platforms these days, the only reason to keep it is legacy compat. If the Vulkan backend is coming very quickly as you say, then I would be in support of putting this in a branch until the rendering backend is solidified; the reason I wanted to do it in GL is because I was under the impression that the Vulkan backend was a long ways off.
If the VK backend is indeed coming soon, I would be happy to write it in Vulkan and keep the object selection in a branch, or in my fork, until the move to Vulkan is complete.
Good discussion, guys.
Let's take the long-term view. We are in an enviable position where we don't have any short-term deadlines or requirements.
As Andrew mentioned, I am on MacOS and I do want the system to run on MacOS/Linux/Windows, so Vukan and Metal seem to be the way to go.
As a high-level view of where I imagine the interactive graphics going, I want to be on the same level as Maya/Houdini/Blender. Though it is early days for kons-9, I want it to be a contender in both GUI features and performance.
Currently, our OpenGL code is in src/graphics/opengl/opengl.lisp and src/graphics/glfw/application-widgets.lisp. I'm hoping porting to Vukan will be as simple as replacing the functions in those files.
I did read the Open Inventor docs recently, and liked much of what I saw.
I can see us having various picking abstractions for manipulators, surface representations, and volumetrics. And probably more that I'm not aware of.
Separating interface from implementation might help towards that end, such as making 3D picking an abstraction (e.g. a class).
My initial intent was a fairly simple abstraction over GPU queries and query pools, similar to Blender's object selection API, but with fewer layers of indirection since we do not need to fit the API into as complex of a system as blender.
Sounds like a plan. I hereby delegate the design to you. :)
Cool. I will post an update on the design here once I have a demo running in Vulkan.
Yeah, I plan on using krma as a base for this. If I have trouble with setup, can I come to you with questions? For actual usage, I can probably make do reading the code.
@nsmoker Have you had a chance to look into this? No rush, just wondering.