Basic GUI Implimentation based on BillFSmith's TilingZoeDepth.
Python 3.10
- If you don't have an NVIDIA GPU, you'll be defaulted to use CPU for depth map generation (It's really slow).
- run
pip3 -r requirements.txt
first to download the packages. - On first start, run
python Tiled_ZoeDepth.py
to download the ZoeDepth repo.
"You can choose to use ZoeD_N for indoor scenes, ZoeD_K for outdoor road scenes, and ZoeD_NK for generic scenes"
source: isl-org/ZoeDepth#10 (comment)
alternatively, click on the drag-and-drop area to select your images.
3. During depth map generation, you will be able to see the original version of Zoe Depth already displayed while the High Quality Depth Map (Tiled Zoe Depth) is processing. You can also see the progress bar moving to ensure user does not panic if it's stuck.
How the progress bar works
The progress bar is more of an indicator on how close the process is done. I placed checkpoints around the code that will closely resemble the process progress. (It's more of a guestimation).
Here are the checkpoints:
1%: Loading Model
- If model is not downloaded, progress bar switches to indeterminate while it's downloading in the background. (At the moment, I have no idea on how to impliment a progressbar download without modifying the torch module.)
10%: Checking if CUDA is available or not
20%++: Generating low-res depth map
30%++: Generating filters
60%: Saving filters
60%++: Compiling tiles & creating depth maps
80%: Saving tiles
90%: Combining depth maps
95%: Generating High-quality depth map
99%: Saving depthmaps...
100%: Images saved
4. After the high quality depth map has been generated, there will be 3 seconds of preview before you can choose to save the image/s or cancel it.
- I only tested this on Windows. Not sure if it will work on Linux or MacOS
Images used:
v3 has a more reliable upload system for larger files. Can take multiple files at once:
https://colab.research.google.com/drive/1Wi-1Ji_fhcoGpK-drT4dVrl5AjfVUQ5M
v2 has a GUI and STL generation:
https://colab.research.google.com/drive/1wbbXpMC_UUwE3e7Tifq9fYNnd5Rn0zna
v1 is broken into sections:
https://colab.research.google.com/drive/1taTL_8GVx1G93ZXp_o-s4FLL-SY6N8TC
This is an adapted version of https://colab.research.google.com/github/isl-org/ZoeDepth/blob/main/notebooks/ZoeDepth_quickstart.ipynb
Corresponding paper : ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth
Here, higher resolution depth maps are generated from the following process:
- Generate a depth map for the overall image
- Split original image into overlapping tiles
- Generate depth maps for the tiles
- Reassemble into a single depth map by applying gradient masks and average weighting from first depth map
- Repeat steps 2-4 at higher resolution
- Combine all three depth maps by:
a) Calculate edge filter from original RGB image
b) Blur edge filter and use as mask for high resolution depth map
c) Apply masked high resolution to average of low and medium resolution depth maps
The difference between the low resolution original and the new version can be seen below: