How to obtain the positions of various components in a Live2D model
Xwdit opened this issue · comments
Hello, I deeply appreciate your efforts in developing this extension. I'm curious to know if there's currently a method to determine the positions or boundaries of each component within Live2D models. I'm aiming to implement a feature similar to what's seen in VTube Studio, where specific content can be anchored to move with a particular part of the Live2D model. Would you have any insights or guidance on this?
What you want to realize is the "item" function in VTuber Studio?
I am not familiar with VTuber Studio, but it would be fun to make it happen!
For example, I can think of a way to get the mesh information of the target component and have it follow the vertex coordinates. This assumes that the mesh structure of the target component does not change with motion.
As for the mesh information, I think we can pass the information taken from the Live2D SDK.
However, this alone will only get the position, but not the orientation or scale information.
The reason for this is that Live2D's Motion results are always given the coordinates of the calculation results.
If you want to do something about this, you may be able to use the following method for pseudo-calculation.
- Determine two specific vertices and calculate a vector from them each time a motion is applied.
- Compare the original vectors with the newly computed vectors and consider the orientation as rotation and the length as scale.
You never know until you try...
I am not able to respond immediately, but I would like to extend GDCubismUserModel to be able to retrieve vertex information.
@Xwdit
I've attached a Godot icon on the head to the Live2D model.
Is this what you want to achieve?
ss.mov
I've attached a Godot icon on the head to the Live2D model.
Is this what you want to achieve?
ss.mov
Yes! That's exactly the effect I was looking for! I apologize for the late response, and I truly appreciate the effort you put into this!
However, I've observed that in the sample video, the Godot icon doesn't appear to rotate in sync with the mesh. Was this aspect not showcased in this video?
Added get_meshes function to the latest commit of v0.1.
In the video, Sprite2D is placed on an appropriate vertex table, but it can be rotated and scaled if calculated.
However, I don't have enough knowledge of mathematics.
I made the following based on the information on the Internet, but it may not work well with the Live2D model you use.
- Rebuild GDCubism with the latest version of 0.1.
- Set the image to Sprite2D, and be sure to set it to a child of GDCubusismEffectCustom.
- This code assigns a script to GDCubismEffectCustom and uses it.
Here's what SceneTree looks like.
The following is a GDScript assigned to GDCubismEffectCustom
extends GDCubismEffectCustom
var first_time: bool = true
var counter: float = 0.0
var base_vct: Vector2
var base_scale: Vector2
@export var art_mesh_name: String = "ArtMesh278"
@export var vector_index_0: int = 0
@export var vector_index_1: int = 1
func _cubism_init(model: GDCubismUserModel):
counter = 0.0
first_time = true
func _cubism_term(model: GDCubismUserModel):
pass
func _cubism_prologue(model: GDCubismUserModel, delta: float):
pass
func _cubism_process(model: GDCubismUserModel, delta: float):
pass
func _cubism_epilogue(model: GDCubismUserModel, delta: float):
var dict_mesh: Dictionary = model.get_meshes()
if dict_mesh.has(art_mesh_name) == false:
return
var ary_mesh: ArrayMesh = dict_mesh[art_mesh_name]
var ary_surface: Array = ary_mesh.surface_get_arrays(0)
var mesh_vertex: PackedVector2Array = ary_surface[ArrayMesh.ARRAY_VERTEX]
if first_time == true:
var fr = ary_surface[ArrayMesh.ARRAY_VERTEX][vector_index_0]
var to = ary_surface[ArrayMesh.ARRAY_VERTEX][vector_index_1]
base_vct = (to - fr)
base_scale = $Sprite2DItem.scale
first_time = false
var fr = ary_surface[ArrayMesh.ARRAY_VERTEX][vector_index_0]
var to = ary_surface[ArrayMesh.ARRAY_VERTEX][vector_index_1]
var calc_vct = (to - fr)
var scale: float = calc_vct.length() / base_vct.length()
$Sprite2DItem.position = fr
$Sprite2DItem.rotation = base_vct.normalized().cross(calc_vct.normalized())
$Sprite2DItem.scale = base_scale * scale
@Xwdit
I have added what you want to achieve to the v0.1.0 sample (demo_effect_custom_01.tscn).
You will need some knowledge of vector calculations and Live2D data structures to understand it, but please try it out if you are interested.
As a week has elapsed, we will now proceed to close this issue.
@Xwdit I have added what you want to achieve to the v0.1.0 sample (demo_effect_custom_01.tscn). You will need some knowledge of vector calculations and Live2D data structures to understand it, but please try it out if you are interested.
Apologies for the late reply, I will give your example a try. Thank you very much for all the work you have done!