vladmandic / human-three-vrm

Realtime VRM Humanoid Avatar Animation using Human Library and ThreeJS

VR Humanoid Avatar using Human library

Head, face, eye, body and hand tracking

Implementation in TypeScript using Human library for human analysis, three for 3D display and scene management and @pixiv/three-vrm for VR model mapping

Notes

VR Model

• Supports any GLTF VRM model
• Examples: https://hub.vroid.com/en

Face

• Implemented: 3D head angle, eye blinks, eye gaze direction, simple emotions, mouth opens

Body

• Implemented: shoulder lean, positions for elbow, wrist, hip, knee
• Missing: front/back detection, detailed leg and arm positions, input validation to avoid unnatural movement

Hands

• Based on calculating finger curl instead of updating positions
• Implemented: hand rotation, finger curls
• Missing: finger positions

Installation

Install using npm install
Run using npm run dev to build a JS bundle and start an internal http/https dev server

2021-09-16 09:50:30 INFO:  human-vrm version 0.2.1
2021-09-16 09:50:30 INFO:  User: vlado Platform: linux Arch: x64 Node: v16.8.0
2021-09-16 09:50:30 INFO:  Application: { name: 'human-vrm', version: '0.2.1' }
2021-09-16 09:50:30 INFO:  Environment: { profile: 'development', config: 'build.json', tsconfig: true, eslintrc: true, git: true }
2021-09-16 09:50:30 INFO:  Toolchain: { build: '0.4.1', esbuild: '0.12.28', typescript: '4.4.3', typedoc: '0.21.9', eslint: '7.32.0' }
2021-09-16 09:50:30 INFO:  Build: { profile: 'development', steps: [ 'serve', 'watch', 'compile' ] }
2021-09-16 09:50:30 STATE: WebServer: { ssl: false, port: 8000, root: '.' }
2021-09-16 09:50:30 STATE: WebServer: { ssl: true, port: 8001, root: '.', sslKey: 'node_modules/@vladmandic/build/cert/https.key', sslCrt: 'node_modules/@vladmandic/build/cert/https.crt' }
2021-09-16 09:50:30 STATE: Watch: { locations: [ 'src/**', 'src/**' ] }
2021-09-16 09:50:31 STATE: Compile: { name: 'human-vrm', format: 'esm', platform: 'browser', input: 'src/human-vrm.ts', output: 'dist/human-vrm.esm.js', files: 2, inputBytes: 17217, outputBytes: 3780124 }
2021-09-16 09:50:31 INFO:  Listening...

Navigate to https://localhost:10031

...Or use sources in your build environment and deploy on your web server