How to set the modal loss rate

[wupeng1989]

Thank you very much for your work, may I ask how the following code miss_2 and miss_1 are set and what they represent. When executing the code, if you set mr=0, theoretically it should be complete multimodal, but actually the code executes into the missing two modes

# forward miss_2 = [0.1, 0.2, 0.3, 0.5, 0.7, 0.9, 1.0] miss_1 = [0.1, 0.2, 0.3, 0.2, 0.1, 0.0, 0.0] if miss_two / (np.round(len(dataloader['train']) / 10) * 10) < miss_2[int(self.args.mr*10-1)]: # missing two modal outputs = model(text, audio, vision, num_modal=1) miss_two += 1 elif miss_one / (np.round(len(dataloader['train']) / 10) * 10) < miss_1[int(self.args.mr*10-1)]: # missing one modal outputs = model(text, audio, vision, num_modal=2) miss_one += 1 else: # no missing outputs = model(text, audio, vision, num_modal=3)

[wupeng1989]

@mdswyz Looking forward to your reply，thanks！

[mdswyz]

Thanks for your attention! First, miss_2 represents the percentage of samples missing 2 modalities and miss_1 represents the percentage missing 1 modality. I take MR=0.1 as an example, let the percentage of samples missing 2 modalities be a, the percentage of samples missing 1 modality be b, and the overall missing rate be 0.1, then we can get 2a+b=3x0.1=0.3, we can see that a and b satisfy the equation:2a+b=0.3, and in the code, we choose a=0.1 and b=0.1 to carry out the experiment of MR=0.1.