Training Faster-RCNN using scientific data

[Zihan Jiang]

We have a special dataset about extreme weather. The dataset has 16 channels and the size of the image is 1000*700px, which is totally different from the ImageNet. We want to implement end to end training based on this dataset.
The Faster-RCNN is composed of three parts: base network + RPN + RCNN. The base network usually is a pre-trained CNN(e.g. ResNet , VGG) for extracting features, but only ImageNet-based pre-trained model can be found because our dataset is not common. So, the question is can we implement end to end training without a pre-trained base network? Or, does the end-to-end training of Faster-RCNN include the parameters in the base network? I have seen many works about end-to-end training, but they all use the pre-trained model as their base network and seems only train the RPN and RCNN.