This repo contains code for the paper Deep learning enables image-based tree counting, crown segmentation and height prediction at national scale
-
Tree crown delineation data now available here
-
Please cite our paper if you find the data useful :)
-
Acknowledgements to Styrelsen for Dataforsyning og Infrastructur and National Land Survey of Finland for open sourcing aeiral images for building the datasets
For the Danish dataset:
-
We offer image patches preprocessed in two ways: a. patch-normalization to 0 mean and unit std (used in paper); b. raw patches with orginial pixel intensities.
-
There are several empty patches with no crown delineations (used as negative sample for training), which can be removed
-
Coordinates have been removed
https://drive.google.com/file/d/1ZNibrh6pa4-cjXLawua6L96fOKS3uwbn/view?usp=sharing
Model names:
trees - date - time - optimizer - lr - input bnads - input shape - no. training tiles - segmentation loss - density loss - boundary weight (for separate individual trees) - model . h5
Depending on your GPU driver, you may try the following environment.yml files to set up the conda environment:
- To set up the same environment:
conda env create -f environment_trees_updated.yml
- If the above does not work, try the following to set up the basic environment and install the missing packages manually:
conda env create -f environment_trees_updated_basic.yml
conda activate tf2151full
Get ready:
-
Aerial images of arbitrary size, preferably with comparable spatial resolution as the training data (20 cm, otherwise may need downsampling or upsampling)
-
A proper model downloaded from the link above, or fine-tuned on local data. select a proper model based on the color bands
--- 🔖 set configs ---
config/RasterAnalysis.py
python main4_large_scale_inference_transfer_other_data.py
Example prediction can be found in /example_1km_tile_tif/, where a model trained on RGB bands was applied to a 1 km x 1 km tile of aerial image captured in Denmark. You may reproduce the results by running step4_large_scale_inference_transfer_other_data.py with the provided configs (config/RasterAnalysis.py).
-
Standardize input image patches channel-wise -> ((image-mean)/std)
-
For large image, predict with overlap (stride < patch width), take max prediction where overlap
-
Upsample / Downsample to match the trained input resolution (20 cm)
-
Finetune with small local annotation dataset
Figure 0. Preparing your own tree crown annotation dataset. Delineate tree crowns inclusively within each selected annotating area.
Get ready:
- A proper model downloaded from the link above, matching the color bands of your local data
- Your small local annotation dataset, prepared in the same structure as the Denmark/Finland dataset (downloadable with the link above). To convert from shapefile/geopackage to raster images, check main0_preprocessing.py
- Recommended: the pretraining data (Denmark dataset), downloadable with the link above
- A correct environment set up with the yml file
--- 🔖 set configs --- config/UNetTrainingFinetune.py
python main1-2_segcount_transfer_learning.py
Convert tree crown delineation data (in shapefile or geopackage format) to raster images for training
python main0_preprocessing.py
--- 🔖 set configs ---
config/Preprocessing.py
python main1_multitask_counting_segmentation.py
--- 🔖 set configs ---
case1: same input spatial resolution: use config/UNetTraining.py
case2: inputs of different spatial resolution (only support 1/2 coarser resolution for now): use config/UNetTraining_multires.py
--- 🔖 example data for demonstration ---
example input data in: example_extracted_data/
--- ✨ major tunable hyperparameters ---
-
boundary weights: determine the penalty on crown boundaries for better individual separation
-
task_ratio: the ratio for weighting the two losses to balance the emphasis on both tasks during training, may differ from case to case
-
normalize: ratio to augment data by randomly normalizing image patches
python main2_height_prediction.py
--- 🔖 set configs ---
config/UNetTraining_CHM.py
python main3_predict_segmentation_counting.py
--- 🔖 set configs ---
config/Model_compare_multires.py
--- 🎏 Example prediction ---
See /example_extracted_data/
-
segmentation result: seg_41.tif
-
counting result: density_41.tif
Figure 1. Overview of the framework used to count individual trees and predict their crown area and height. a, Deep learning-based framework for individual tree counting, crown segmentation, and height prediction. Spatial locations of individual trees are incorporated in the tree density maps and the crown segmentation maps. The canopy height map (CHM) derived from LiDAR data provides pixel-wise height information, which, when available for a specific study area, can optionally be used as an additional input band for the individual tree counting and crown segmentation tasks. b, Data preparation and modeling for tree counting and crown segmentation. The manually delineated individual tree crowns are modeled as density maps for the counting task by extracting the polygon centroids. The gaps between adjacent crowns are highlighted for the separation of individual tree crowns during the training phase.
Figure 2. Example products from the proposed framework. a, Wall-to-wall tree count prediction for Denmark. b, Detailed examples showing the individual tree counting (second row), crown segmentation (third row), and height prediction (third row) from three major types of landscapes (deciduous forest, coniferous forest, and non-forest). c, Large-scale individual tree crown segmentation results colored by height predictions. Examples in b and c were sampled from the region indicated by the orange box in a.