PhenoBench is a large dataset and benchmarks for the semantic interpretation of images of real agricultural fields. The benchmark tasks cover semantic segmentation of crops and weeds, panoptic segmentation of plants, leaf instance segmentation, detection of plants and leaves, and the novel task of hierarchical panoptic segmentation for jointly identifying plants and leaves.
In this repository, we provide code and instructions to reproduce the baseline results reported in our paper. Please see the sub-folders of the individual tasks for further instructions, configurations, and setup of the baselines.
Important: These are copies from the original repositories. We tried to make it transparent where the code originated with a link to the original repository. Thus, leave a ⭐ at their repository if you use their code.
To allow to produce the exact same results as reported in the paper, we also provide pre-trained models for our baseline methods. Thus, one can use the inference code to reproduce the results. For an overview of the pre-trained models, i.e., checkpoints of the final models.
Checkpoints
| Task | Approach | Checkpoint |
|---|---|---|
| Semantic Segmentation | ERFNet | Download |
| Semantic Segmentation | DeepLabV3+ | Download |
| Panoptic Segmentation | Mask R-CNN | Download |
| Panoptic Segmentation | Panoptic DeepLab | Download |
| Panoptic Segmentation | Mask2Former | Download |
| Leaf Instance Segmentation | Mask R-CNN | Download |
| Leaf Instance Segmentation | Mask2Former | Download |
| Hierarchical Panoptic Segmentation | Weyler et al. | Download |
| Hierarchical Panoptic Segmentation | HAPT | Download |
| Plant Detection | Faster R-CNN | Download |
| Plant Detection | Mask R-CNN | Download |
| Plant Detection | YOLOv7 | Download |
| Leaf Detection | Faster R-CNN | Download |
| Leaf Detection | Mask R-CNN | Download |
| Leaf Detection | YOLOv7 | Download |
For comparison and rendering of detailed results, we also provide the submissions files of the predictions to the individual CodaLab competitions. These can be used with the code in the PhenoBench development kit to visualize the results of the baseline and could be valuable for additional images and qualitative comparisons in papers.
Please refer to the PhenoBench development kit for further details on the visualization of the results and additional information on the usage of the provided tools.
Predictions
| Task | Approach | Validation | Test |
|---|---|---|---|
| Semantic Segmentation | ERFNet | Download | Download |
| Semantic Segmentation | DeepLabV3+ | Download | Download |
| Panoptic Segmentation | Mask R-CNN | Download | Download |
| Panoptic Segmentation | Panoptic DeepLab | Download | Download |
| Panoptic Segmentation | Mask2Former | Download | Download |
| Leaf Instance Segmentation | Mask R-CNN | Download | Download |
| Leaf Instance Segmentation | Mask2Former | Download | Download |
| Hierarchical Panoptic Segmentation | Weyler et al. | Download | Download |
| Hierarchical Panoptic Segmentation | HAPT | Download | Download |
| Plant Detection | Faster R-CNN | Download | Download |
| Plant Detection | Mask R-CNN | Download | Download |
| Plant Detection | YOLOv7 | Download | Download |
| Leaf Detection | Faster R-CNN | Download | Download |
| Leaf Detection | Mask R-CNN | Download | Download |
| Leaf Detection | YOLOv7 | Download | Download |
Please see the licenses of the particular sub-folders.
If you use the specific baseline code, please cite the corresponding paper. We added a file CITATION.md that includes the suggested BibTeX entry for the baseline code.
If you use our dataset, then you should cite our paper PDF:
@article{weyler2023dataset,
author = {Jan Weyler and Federico Magistri and Elias Marks and Yue Linn Chong and Matteo Sodano
and Gianmarco Roggiolani and Nived Chebrolu and Cyrill Stachniss and Jens Behley},
title = {{PhenoBench --- A Large Dataset and Benchmarks for Semantic Image Interpretation
in the Agricultural Domain}},
journal = {arXiv preprint},
year = {2023}
}