auto_lightgbm not working

Question

auto_lightgbm not working

jarichb opened this issue 3 months ago · comments

Describe the bug
auto_lightgbm is not working, when following the documentation: https://docs.functime.ai/forecasting/#quantile-regression

To Reproduce

import json
from timeit import default_timer

import polars as pl

from functime.cross_validation import train_test_split
from functime.forecasting import auto_linear_model, linear_model, naive, snaive, auto_lightgbm
from functime.metrics import smape
from functime.preprocessing import scale
from functime.seasonality import add_fourier_terms

start_time = default_timer()

y = pl.read_parquet(
    "https://github.com/functime-org/functime/raw/main/data/commodities.parquet"
)
entity_col, time_col = y.columns[:2]
X = y.select([entity_col, time_col]).pipe(add_fourier_terms(sp=12, K=6)).collect()

print("🎯 Target variable (y):\n", y)
print("📉 Exogenous variables (X):\n", X)

test_size = 3
freq = "1mo"
y_train, y_test = train_test_split(test_size)(y)
X_train, X_test = train_test_split(test_size)(X)

y_pred_10 = auto_lightgbm(alpha=0.1, freq="1d")(y=y_train, fh=28)
y_pred_90 = auto_lightgbm(alpha=0.9, freq="1d")(y=y_train, fh=28)

Expected behavior
that the example in the documentation would at least execute

Screenshots

Desktop (please complete the following information):

OS: Mac
Functime version: 0.9.5.

Additional context
As per requested on LinkedIn

baggiponte · Answer 1 · Wed Apr 17 2024 03:02:10 GMT+0800 (China Standard Time)

Ciao @jarichb, thanks for reporting and sorry for the late reply. I see the error! Did you make sure to install functime[lgb]?

We documented that on this page: how can we improve that?

I think I need to make this error be raised sooner.

jarichb · Answer 2 · Wed Apr 17 2024 03:19:56 GMT+0800 (China Standard Time)

interestingly enough, I can't seem to get it installed like that either, see error below.
It does mention requirement already satisfied but it still can't do the above function, so it seems like it doesn't install it properly.

UPDATE: this error was resolved with 'brew install libomp', and is a LGB specific thing, now I got it to work!
I do think that 'ImportError' not callable is a rather vague error though, if it could display to make the specific install, I feel like that would help.

(.env) jarich@Jarichs-Air functime % pip install "functime[lgb]"

Requirement already satisfied: functime[lgb] in ./.env/lib/python3.8/site-packages (0.9.5)
Requirement already satisfied: cloudpickle in ./.env/lib/python3.8/site-packages (from functime[lgb]) (3.0.0)
Requirement already satisfied: flaml<3,>=2.0.2 in ./.env/lib/python3.8/site-packages (from functime[lgb]) (2.1.2)
Requirement already satisfied: holidays in ./.env/lib/python3.8/site-packages (from functime[lgb]) (0.46)
Requirement already satisfied: numpy in ./.env/lib/python3.8/site-packages (from functime[lgb]) (1.24.4)
Requirement already satisfied: polars>=0.20.7 in ./.env/lib/python3.8/site-packages (from functime[lgb]) (0.20.21)
Requirement already satisfied: scikit-learn<2,>=1.2.2 in ./.env/lib/python3.8/site-packages (from functime[lgb]) (1.3.2)
Requirement already satisfied: scipy in ./.env/lib/python3.8/site-packages (from functime[lgb]) (1.10.1)
Requirement already satisfied: tqdm in ./.env/lib/python3.8/site-packages (from functime[lgb]) (4.66.2)
Requirement already satisfied: typing-extensions in ./.env/lib/python3.8/site-packages (from functime[lgb]) (4.11.0)
Collecting lightgbm (from functime[lgb])
  Using cached lightgbm-4.3.0.tar.gz (1.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: joblib>=1.1.1 in ./.env/lib/python3.8/site-packages (from scikit-learn<2,>=1.2.2->functime[lgb]) (1.4.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./.env/lib/python3.8/site-packages (from scikit-learn<2,>=1.2.2->functime[lgb]) (3.4.0)
Requirement already satisfied: python-dateutil in ./.env/lib/python3.8/site-packages (from holidays->functime[lgb]) (2.9.0.post0)
Requirement already satisfied: six>=1.5 in ./.env/lib/python3.8/site-packages (from python-dateutil->holidays->functime[lgb]) (1.16.0)
Building wheels for collected packages: lightgbm
  Building wheel for lightgbm (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for lightgbm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      2024-04-16 21:20:24,534 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/cmake/data/bin/cmake --version
      2024-04-16 21:20:24,540 - scikit_build_core - INFO - CMake version: 3.29.2
      *** scikit-build-core 0.8.2 using CMake 3.29.2 (wheel)
      2024-04-16 21:20:24,545 - scikit_build_core - INFO - Build directory: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build
      *** Configuring CMake...
      2024-04-16 21:20:24,555 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/ninja/data/bin/ninja --version
      2024-04-16 21:20:24,646 - scikit_build_core - INFO - Ninja version: 1.11.1
      2024-04-16 21:20:24,647 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/cmake/data/bin/cmake -S. -B/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build -DCMAKE_BUILD_TYPE:STRING=Release -C/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build/CMakeInit.txt -DCMAKE_MAKE_PROGRAM=/private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/ninja/data/bin/ninja -D__BUILD_FOR_PYTHON:BOOL=ON
      loading initial cache file /var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build/CMakeInit.txt
      -- The C compiler identification is AppleClang 12.0.5.12050022
      -- The CXX compiler identification is AppleClang 12.0.5.12050022
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
      -- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES)
      -- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND)
      -- Found OpenMP_C: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include
      -- Found OpenMP_CXX: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include
      -- Found OpenMP: TRUE
      -- Performing Test MM_PREFETCH
      -- Performing Test MM_PREFETCH - Failed
      -- Performing Test MM_MALLOC
      -- Performing Test MM_MALLOC - Success
      -- Using _mm_malloc
      -- Configuring done (0.7s)
      -- Generating done (0.0s)
      -- Build files have been written to: /var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build
      *** Building project with Ninja...
      2024-04-16 21:20:25,354 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/cmake/data/bin/cmake --build /var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build
      ninja: error: '/opt/homebrew/opt/libomp/lib/libomp.dylib', needed by '/private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-install-9ka6inly/lightgbm_51586f541a1746beb85cf9098a705212/lib_lightgbm.so', missing and no known rule to make it
      
      *** CMake build failed
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for lightgbm
Failed to build lightgbm
ERROR: Could not build wheels for lightgbm, which is required to install pyproject.toml-based projects

baggiponte · Answer 3 · Wed Apr 17 2024 23:48:43 GMT+0800 (China Standard Time)

UPDATE: this error was resolved with 'brew install libomp', and is a LGB specific thing, now I got it to work!
I do think that 'ImportError' not callable is a rather vague error though, if it could display to make the specific install, I feel like that would help.

Yes, I should explicitly mark that lightgbm requires libomp. Will add it to the docs and then proceed to close the issue. Does this sound right to you? 😊

jarichb · Answer 4 · Thu Apr 18 2024 00:23:39 GMT+0800 (China Standard Time)

Yes, sounds good to me! Thank you

…

On Wed, 17 Apr 2024, 17:49 baggiponte, ***@***.***> wrote: UPDATE: this error was resolved with 'brew install libomp', and is a LGB specific thing, now I got it to work! I do think that 'ImportError' not callable is a rather vague error though, if it could display to make the specific install, I feel like that would help. Yes, I should explicitly mark that lightgbm requires libomp. Will add it to the docs and then proceed to close the issue. Does this sound right to you? 😊 — Reply to this email directly, view it on GitHub <#190 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP4NCDU56APNWAC2XE7XJ5DY52KXBAVCNFSM6AAAAABF5IZGVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRRGYYTOOJRGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

baggiponte · Answer 5 · Thu Apr 18 2024 17:00:26 GMT+0800 (China Standard Time)

Ciao @jarichb! The preview of the new docs is here: I'll merge the request as soon as you give it a green light 😊