mAP results inconsistent

[zzyy410]

helllo.Why are the mAP results inconsistent every time I train under the same conditions?code is rtdetr_pytorch? What should I fix, and could you provide some advice,please

[lyuwenyu]

This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.

You can try fixing random seeds (eg. numpy.random.seed random.seed and torch.manual_seed)to obtain consistent results.

[zzyy410]

thanks！

This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.

You can try fixing random seeds (eg. numpy.random.seed random.seed and torch.manual_seed)to obtain consistent results.

thanks！

[zzyy410]

This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.

You can try fixing random seeds (eg. numpy.random.seed random.seed and torch.manual_seed)to obtain consistent results.

Hi again sorry to bother you, I've tried setting seed before training starts but still can't get consistent results for each target detection, where should I set the random seed? Is it easy to tell which file? It would be more appreciated if you can tell the exact location. Thank you for your time!
Here's where I've tried to set random seeds:

class DetSolver(BaseSolver):
def fit(self, ):
print("Start training")
self.train()
# add myself 可复现
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
......
def val(self, ):
self.eval()
# add myself 可复现
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# dist.set_seed(42)
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
.......
def train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module,
data_loader: Iterable, optimizer: torch.optim.Optimizer,
device: torch.device, epoch: int, max_norm: float = 0, **kwargs):
# add myself 可复现
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# dist.set_seed(42)
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
.......
def evaluate(model: torch.nn.Module, criterion: torch.nn.Module, postprocessors, data_loader, base_ds, device, output_dir):
# add myself 可复现
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# dist.set_seed(42)
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
.......
if name == 'main':
parser = argparse.ArgumentParser()
# parser.add_argument('--config', '-c', type=str, )
parser.add_argument('--config', '-c', type=str, default="/home/lw/rtdetr_pytorch/configs/rtdetr/rtdetr_r18vd_6x_sonar.yml")
parser.add_argument('--resume', '-r', type=str, default=False)
parser.add_argument('--tuning', '-t', type=str, default=False)
parser.add_argument('--test-only', action='store_true', default=False,)
parser.add_argument('--amp', action='store_true', default=False,)
args = parser.parse_args()
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
main(args)

[charles]

You can add it into first line of main function

def set_seed_and_config():
    np.random.seed(42)
    torch.manual_seed(42)
    torch.backends.cuda.matmul.allow_tf32 = True
    torch.backends.cudnn.benchmark = True
    torch.backends.cudnn.deterministic = False

[worldNebula]

This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.
You can try fixing random seeds (eg. numpy.random.seed random.seed and torch.manual_seed)to obtain consistent results.

Hi again sorry to bother you, I've tried setting seed before training starts but still can't get consistent results for each target detection, where should I set the random seed? Is it easy to tell which file? It would be more appreciated if you can tell the exact location. Thank you for your time! Here's where I've tried to set random seeds:

class DetSolver(BaseSolver): def fit(self, ): print("Start training") self.train() # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False np.random.seed(42) random.seed(42) torch.manual_seed(42) ...... def val(self, ): self.eval() # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False # dist.set_seed(42) np.random.seed(42) random.seed(42) torch.manual_seed(42) ....... def train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module, data_loader: Iterable, optimizer: torch.optim.Optimizer, device: torch.device, epoch: int, max_norm: float = 0, **kwargs): # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False # dist.set_seed(42) np.random.seed(42) random.seed(42) torch.manual_seed(42) ....... def evaluate(model: torch.nn.Module, criterion: torch.nn.Module, postprocessors, data_loader, base_ds, device, output_dir): # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False # dist.set_seed(42) np.random.seed(42) random.seed(42) torch.manual_seed(42) ....... if name == 'main': parser = argparse.ArgumentParser() # parser.add_argument('--config', '-c', type=str, ) parser.add_argument('--config', '-c', type=str, default="/home/lw/rtdetr_pytorch/configs/rtdetr/rtdetr_r18vd_6x_sonar.yml") parser.add_argument('--resume', '-r', type=str, default=False) parser.add_argument('--tuning', '-t', type=str, default=False) parser.add_argument('--test-only', action='store_true', default=False,) parser.add_argument('--amp', action='store_true', default=False,) args = parser.parse_args() np.random.seed(42) random.seed(42) torch.manual_seed(42) main(args)

Hello, now I have the same problem as you, how do you solve it?