jadarve / lluvia

A real-time computer vision engine implemented on top of Vulkan API.

Home Page:https://lluvia.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Low Level Unified VIsion Algorithms (LLUVIA)

lluvia.ai

CircleCI

Lluvia is a computer vision engined designed for real-time applications. It uses the Vulkan API to access the GPU and dispatch computations. Users can describe computations as a compute pipeline where nodes are compute shaders running on the GPU.

Real Time Optical Flow

This demo is for my real-time optical flow algorithm running using Lluvia on a GTX-1080 GPU. For images of 1016x544 resolution, computation take around 1 millisecond to complete.

Supported Platforms

  • Linux.
  • Windows.
  • Android (work in progress).

Try it out

Try the examples in Google Colab. Make sure the runtime is configured to GPU.

Workflow

Lluvia uses a compute graph to organize and schedule computations on the GPU. The development workflow circles around coding and debugging nodes in such a graph until the whole algorithm is built:

  1. The node's inputs, outputs and parameters are described in a Lua script . This description will later be used to instantiate nodes in the graph.

  2. The node's computation in the GPU is coded as a compute shader in Open GL Shading Language (GLSL). Shaders are compiled into SPIR-V intemediate representation for later load into the GPU.

  3. The node's description and compute shader are loaded into Lluvia's runtime. After this, nodes can be instantiated to build the compute graph and be dispatched to the GPU.

From a user perspective, one needs to only care about describing nodes (inputs, outputs, compute shader, etc.) and connecting nodes to form a graph. Lluvia takes care of the low-level details of dispatching the graph for execution onto the GPU. This workflow allows porting the compute graph from one platform to another with ease.

License

Apache 2.0. See LICENSE for more details

About

A real-time computer vision engine implemented on top of Vulkan API.

https://lluvia.ai/

License:Apache License 2.0


Languages

Language:C++ 47.6%Language:Cython 15.4%Language:Lua 15.3%Language:Python 10.1%Language:Starlark 5.8%Language:GLSL 5.5%Language:Dockerfile 0.2%Language:C 0.2%Language:Shell 0.1%