Operation CWAL of "...아로나 또한 비서로 개발되어 나올 가능성이 있을까요?".
※ Model files not included
v1demo.mp4
Demonstration of v1 with polyglot module.
Base : Java - libgdx spine runtime
Targeted screen size : 1080p
Get a copy of the spine model from somewhere else, then place it to ./aronares.
※ Implemented method for lipsync is quite different from that of in-game example (which was added much later).
Base : Python - kobart or polyglot
-
kobart
Finetuned KoBART from KoBART-chatbot.
Fast, but generates a fixed answer for a given sentence (encoder-decoder).
To make it work on windows OS, you need to solve an error (as far as I remember, that was something related to type conversion in torch).
-
polyglot
EleutherAI's polyglot-ko-1.3b finetuned with datasets of KoAlpaca.
Slow, even with time limits - can be better if you have a modern gpu
(probably better than my old 970m). Also takes a lot of Vram.To make it work on windows OS, you need bitsandbytes modified for windows and some additional struggles for manual dtype allocations by opening up the library and fixing it.
Base : Python - mb-istft-vits-multilingual
Base : Python - ezTransWeb-omni
There are some clumsy requirements for this. Please refer to https://github.com/FENRlR/ezTransWeb-omni.
- Bug in PMA
- Without PMA, the sprite innates white borders and slight transparancy
- Limited options for the window
Base : C# - SpineViewerWPF
Targeted screen size : 1080p
Get a copy of the spine model from somewhere else, then place it to ./aronares/arona.
※ WIP - now with correct representation of PMA.
- Customized language model with emotion embeddings
- Lipsync options (in-game style amplitude based vs freq based)
- Functional interaction (simple search, opening something, etc. - ideas needed)
- STT (not my style but if needed)
- GPU usage goes up and hits nearly 50% in GTX970m(and 30% with RTX4060ti - half of it comes from dwm) when monitored with task manager, despite of the actual usage of ~14% from gpu profiler
- Not stable (at least in my system) - Turns out to be an overflow exception that comes from a mysterious infinite loop of mixblend while applying a state to the skeleton. Disabling it suppressed the issue, but I would not call this an ideal solution.