Giters

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

Home Page:https://huggingface.co/docs/transformers.js

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Depth anything in Python

VladimirYugay opened this issue · comments

commented

Question

Amazing demo for the depth-anything!

I want to have a similar point cloud, but in Python, and wondering what's the logic behind your js implementation.

Specifically:

  1. How do you set up the intrinsic matrix and backproject the depth map and color to the 3D space?
  2. What is the difference between Xenova/depth-anything-small-hf and LiheYoung/depth-anything-small-hf?
commented
  1. Fortunately three.js handles all the complicated 3D scene management for me. All the logic can be found in the setupScene function here. If you want to achieve something similar in python, you can use the gradio 3D model component. You will need to convert the depth image to a glb trimesh (see example code here).
  2. As stated in the README of Xenova/depth-anything-small-hf, it is the same as LiheYoung/depth-anything-small-hf, but with ONNX weights to be compatible with transformers.js (so it can run in the browser).