Imagine a game where AI-powered Non-Player Characters (NPCs) can not only respond to your questions but also perform actions based on your requests! ๐ Cohere's Command model generates character responses, including dialogue and actions, using templates that define their talking style, action list, and bio. ๐ฌ Unreal Engine, even though it can't run Python directly, communicates with it through text files, enabling real-time interaction. ๐ฎ AssemblyAI transcribes player dialogue, while ElevenLabs generates voice dialogue, which is then imported into the game using the Runtime Audio Importer. ๐ฃ๏ธ The MetaHuman SDK brings these characters to life with facial animations. ๐ฎ Actions requested by players are parsed and executed, ranging from animations to item spawns and even AI-generated music creation. ๐น๏ธ
The Repository contains the python script and the demo.umap which you can import in a level in unreal to look at the level blueprint and see the logic. ๐ There are a couple of other files as well that can help you understand better. ๐ If you find this repository helpful, consider giving it a star! โญ
Discord Server: https://discord.gg/CfK7DCWKwy
- ๐ค Cohere's Language Models and Embedding Models
- ๐ฃ๏ธ ElevenLabs
- ๐ AssemblyAI
- ๐๏ธ ChromaDB
- ๐ Langchain
- ๐ฎ Unreal Engine
- ๐ฒ Quixel Bridge
- ๐ง Metahuman SDK
- ๐ Mixamo
- ๐ Runtime Audio Importer
- "Warlock's End - Metahuman Armor" (https://skfb.ly/oHInI) by pdravecky.fx is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).(https://sketchfab.com/3d-models/warlocks-end-metahuman-armor-e3352bc5d74f4c38afe7bdf700420947)
- "Sci-fi suit for Metahuman" (https://skfb.ly/oFuBz) by Alexander Kurmanin is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/). (https://sketchfab.com/3d-models/sci-fi-suit-for-metahuman-0e938256aa4f4af3b3a23c811da585fa#download)
- "Magic Ring - Blue" (https://skfb.ly/6Uq7O) by Noob Model :D is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).(https://sketchfab.com/3d-models/magic-ring-blue-5c81d8a7610543779d0c7e85f9175fae)
- "Magic Ring - Yellow" (https://skfb.ly/6WMOx) by Noob Model :D is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).(https://sketchfab.com/3d-models/magic-ring-yellow-757ff3fbfb2045c68165c06494917faa#download)
- Quixel Bridge (https://quixel.com/bridge)
- Metahuman Creator (https://metahuman.unrealengine.com/)
If you found this interesting check out Alystria AI for more fun projects
https://www.linkedin.com/company/alystria-ai
- ๐ Github: https://github.com/AkshitIreddy
- ๐ค Twitter: https://twitter.com/Akshit2089
- ๐ก LinkedIn: https://www.linkedin.com/in/akshit-ireddy
- โ๏ธ Medium: https://medium.com/@akshit.r.ireddy