yanshil / SysMocap

A real-time motion capture system for 3D virtual character animating.

SysMocap

English Version | 中文版本

A cross-platform real-time video-driven motion capture and 3D virtual character rendering system for VTuber/Live/AR/VR.

Available for Windows, macOS (packaged) & Linux (source code only)

Download Now (zipped, without installation)

(This is a multi-language software, including English. )

Highlights

🌟 Beautiful GUI with Material Design 3 Color System

🌟 Easy to used. You can import 3D models just with drags.

add-model-drag.mp4

🌟 Support WebXR API on Mocap Forwarding (HTTPS only)

webxr-ar-demo.mp4

🌟 Model viewer with bones & dressing controller

🌟 Support OBS live-streaming

🌟 Support full-body motion capture

🌟 Support Auto Skeleton Detection for All VRM files and Mixamo Format FBX files

🌟 Support Any Skeleton Structure with Manual Mapping

🌟 Does not require a discrete graphics card and runs smoothly even on eight-year-old computers (i7-4790k/GTX770/16G RAM)

🌟 Powered by Mediapipe and Kalidokit with Web Technologies

More Effect Demonstration

🌟 Facial

🌟 Half-body

🌟 Half-body with Hands

🌟 Full-body

System architecture

How to use

Run on your computer from source code (need lastest Node.js):

git clone https://github.com/xianfei/SysMocap.git
cd SysMocap
npm i
npm start

Bugs

1. On Windows platform, "Use Discrete Graphics on Dual GPU Laptop" and "Mocap Data Forward" can not use at same time.

2. Camera controller only support VRM

3. Forwarding only support VRM

Notice

1. HTTP & HTTPS will use same port in Mocap Data Forward.

Required Skeleton Node in glTF/glb/FBX Model File for Macap:

(If not same as above, you need rebind them manually.)

• Hips (Main Node, both Position and Rotation. Ratation only for other nodes)

• Neck

• Chest

• Spine

• RightUpperArm

• RightLowerArm

• LeftUpperArm

• LeftLowerArm

• LeftUpperLeg

• LeftLowerLeg

• RightUpperLeg

• RightLowerLeg

Development progress

To-Do

• Settings page and global settings utils

• Add play/pause button and progress bar when mocap from video

• Support bones binding for glTF/glb

• Support rendering glTF/glb model

• Support binding when bones' name is non-uniformed

• Model library add user's custom 3D model

• Live plug-in / interface for Open Broadcast Software

• Output video ( using such as libffmpeg )

• Support per-frame rendering without drop frame

• Support c-s architecture for online video mocap ( on cloud )

• Support Material Designed 3 Color System (color picking)

• Mocap data forwarding via network

• Adapt for Linux and macOS

Thanks

You can see all dependencies in package.json

• google/mediapipe/Holistic for Mocap

• kalidokit for Calulate Mocap Data

• electron and Vue.js for GUI Framework

• Material color utilities for Color Picking