This data set was created to study Deep Learning-based Guidance Line Extraction of Automatic Weeding Equipment in Paddy Field. The images in the dataset consists of two parts, one was manually collected from a paddy field in Guangzhou city in 2019 while the other was collected in Jiangmen city in 2017. The data set has a total of 704 images. The main disturbances in rice seedling detection are Alternanthera philoxeroides and Sagittaria trifolias which are common in rice fields. These images, with original sizes of 3648 * 2048, are from rice seedlings growing for 2 to 3 weeks, which is the best time for weeding. It should be noted that images of single rice seedlings are not included in the dataset, which is different from the traditional seedling detection methods. Each image should include at least two rows of rice seedling in order to improve the robustness, the images with an irregular shape of rice or weeds are also included in the dataset.