How to train the classifier model?

[SuperJay1996]

According the paper and code, the backbone should be frozen for faster training, however, the provided classifier model do not have the same parameters with the provided backbone, it is really confused for training the final classifier model?
The final classifier model only have 0.343 mAP on COCO dataset with "pct_base_woimgguide_classifier.py".
{"mode": "val", "epoch": 210, "iter": 67, "lr": 0.0, "AP": 0.3434, "AP .5": 0.68824, "AP .75": 0.30028, "AP (M)": 0.33237, "AP (L)": 0.36357, "AR": 0.38416, "AR .5": 0.71678, "AR .75": 0.36288, "AR (M)": 0.3646, "AR (L)": 0.41226}.

It will be very helpful if this question could be solved, looking forward to your response.