vame.egocentric_alignment gives: "IndexError: list index out of range"

[WeissShahaf]

I have a a CSV with 10 markers.
the project is succesfully created, but vame.egocentric_alignment errors out

In [1]: import vame
Using CUDA
GPU active: True
GPU used: NVIDIA GeForce RTX 3090

In [2]:

In [2]: config = r'H:\VAME\tempiDir\test2-Mar8-2022\config.yaml'

In [3]: config = 'H:/VAME/tempiDir/test2-Mar8-2022/config.yaml'

In [4]: vame.egocentric_alignment(config, pose_ref_index=[3,4,5,6], crop_size=(300,300), use_video=True, video_format='.mp4', check_video=True)
Aligning data GL999_circl1_cam1_20210708_112_4k_masked, Pose confidence value: 0.99
Compute background image for video GL999_circl1_cam1_20210708_112_4k_masked: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [06:32<00:00, 2.55it/s]
Finishing up!
Align frames: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50160/50160 [2:55:12<00:00, 4.77it/s]

IndexError Traceback (most recent call last)
in
----> 1 vame.egocentric_alignment(config, pose_ref_index=[3,4,5,6], crop_size=(300,300), use_video=True, video_format='.mp4', check_video=True)

H:\VAME\VAME\vame\util\align_egocentrical.py in egocentric_alignment(config, pose_ref_index, crop_size, use_video, video_format, check_video)
311 for file in filename:
312 print("Aligning data %s, Pose confidence value: %.2f" %(file, confidence))
--> 313 egocentric_time_series, frames = alignment(path_to_file, file, pose_ref_index, video_format, crop_size,
314 confidence, use_video=use_video, check_video=check_video)
315 np.save(os.path.join(path_to_file,'data',file,file+'-PE-seq.npy'), egocentric_time_series)

H:\VAME\VAME\vame\util\align_egocentrical.py in alignment(path_to_file, filename, pose_ref_index, video_format, crop_size, confidence, use_video, check_video)
291
292 if check_video:
--> 293 play_aligned_video(frames, n, frame_count)
294
295 return time_series, frames

H:\VAME\VAME\vame\util\align_egocentrical.py in play_aligned_video(a, n, frame_count)
237
238 for c,j in enumerate(n[i]):
--> 239 cv.circle(im_color,(j[0], j[1]), 5, colors[c], -1)
240
241 cv.imshow('Frame',im_color)

IndexError: list index out of range

[K L]

Hi!

I think the problem is in how you specify pose_ref_index. This function takes only two indices like described in the demo.py.:

# Step 1.2:
# Align your behavior videos egocentric and create training dataset:
# pose_ref_index: list of reference coordinate indices for alignment
# Example: 0: snout, 1: forehand_left, 2: forehand_right, 3: hindleft, 4: hindright, 5: tail
vame.egocentric_alignment(config, pose_ref_index=[0,5])

You need two anchor points to align the animal egocentrically. Here, we used the snout and tail base marker, which have are the first (0) and last (5) marker of DLC, respectively.

I hope this helps with understanding the input to this function and you can continue with VAME.

If your data is already egocentrically due to the animal is being (head)fixed, please refer to the csv_to_numpy.py function.

Cheers,
Kevin