As the increase in the world population the demand of the rice is also increases. In order to increase the growth of rice in the rice crop it is necessary to detect the weed and insects in the rice crop to minimize the growth of weed and insects so that the growth of the rice can be increased.Insect and Weed detection is the important factor to be analyzed. Unmanned Air Vehicle (UAV) is used for data acquisition of rice crop in different phases and states so that high quality of RGB images can be captured. In which we have taken 15 different types of rice crop insects species images and different phases of weed images to train the model. The proposed method facilitates the extraction of weed and insects into the rice crop field using deep learning concept faster region-based convolutional neural networks(Faster R-CNNs) it is implemented using Python3 with the help of Tensorflow API. The result shows that Faster R-CNN method is the state of arts method for detection and classification of weed and insects with good accuracy rate.