LightGBM error when running examples/sentinel2-cloud-detector-example.ipynb

[Roberto Tatis-Muvdi]

Dear all

When trying to run the example notebook at examples/sentinel2-cloud-detector-example.ipynb I'm getting the following error:

[LightGBM] [Fatal] Model format error, expect a tree here. met 024 136899 30508 15753 11580 148560 11243 243772 16949 7857 8046 69628 47646 20778 76112 71160 8759 5305 35855 34350 23939 57599 11800 7850 48513 25948 22317 6019 4505 16298 14114 6677 24517 14271 10438 10246 7507 5657 8796 40623 22481 14432 4415 36676 41681 17124 24557 10270 12513 58880 88121 8489 11322 102437 44832 57605 47779 43914 37606 53154 15757 14291 10335 12297 124586 84162 18401 3516 41548 65279 7868 38090 43578 90421 37911 5851 20744 9941 90596

I'm using:

• Windows 10
• python 3.8.8
• numpy 1.20.1
• sentinelhub 3.2.0
• s2cloudless 1.5.0
• lightgbm 3.1.1

Any hints are greatly appreciated

Roberto

[Matej Aleksandrov]

Hi @robetatis,

I'm not able to reproduce problems on Linux, but I have a few more questions:

• Does the error happen in this line where lightgbm.Booster tries to load a LightGBM model that has been serialized into a text file?

• What kind of environment are you using? Did you install packages with Conda or pip?

• Does the same error happen if you try using an older version of lightgbm package?

• Have you perhaps checked in LightGBM GitHub repository for any similar issues? For example, this one seems very similar.

[Roberto Tatis-Muvdi]

Hi @AleksMat,

Thanks for your reply.

• LightGBM does not expose the call stack. I'm only seeing the error message I posted before
• I created a conda env for the basic stuff and, for s2cloudless I used pip install git+git://github.com/sentinel-hub/sentinel2-cloud-detector.git
• I'll downgrade LightGBM and see what happens
• I was hoping to use directly the functions in s2cloudless without having to dive into LightGBM

I also tried a quick test with artificial data, getting the exact same error (including the numbers):

import numpy as np
from s2cloudless import S2PixelCloudDetector

bands = np.random.uniform(0, 10000, (10, 10, 10))
cloud_detector = S2PixelCloudDetector(threshold=0.4, average_over=4, dilation_size=2)
cloud_probs = cloud_detector.get_cloud_probability_maps(bands)

[Matej Aleksandrov]

Before installing s2cloudless with pip I recommend you try installing lightgbm package with conda. The latest version is available on both anaconda and conda-forge channels.

In the end, s2cloudless is just a wrapper around lightgbm package and a cloud detection model that has been exported in a native LightGBM text format. Because of that I suspect that something is wrong with your installation of lightgbm.

[Roberto Tatis-Muvdi]

Hi @AleksMat,
Re-building the env and installing everything from conda-forge solved it.
Thanks!