functime-org / functime

Time-series machine learning at scale. Built with Polars for embarrassingly parallel feature extraction and forecasts on panel data.

Home Page:https://docs.functime.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error in transform_new Function Due to Index Column Mismatch in Scale Transformer

jhug12 opened this issue · comments

Describe the bug
The transform_new function of the scale transformer in the functime preprocessing module fails during execution. This issue arises when the function attempts to join the X DataFrame with the _mean DataFrame based on index columns (idx_cols) specified as 'time' and 'entity'. The join operation fails because _mean contains only the 'entity' column and lacks the 'time' column, leading to a key mismatch and an execution error.

To Reproduce

import polars as pl
from functime.preprocessing import scale

# Load sample data
y = pl.read_parquet("https://github.com/functime-org/functime/raw/main/data/commodities.parquet")

# Initialize the scale transformer
transformer = scale()

# Fit the transformer
_ = y.pipe(transformer).collect()

# Attempt to transform new data and observe the error
transformer.transform_new(y).collect()

Expected behavior
The function would only join on columns that are present in _mean.

Desktop (please complete the following information):

  • OS: Ubuntu 22.04.4 LTS
  • Python version: 3.8.19
  • functime version: 0.9.5

Ciao and thanks for the bug report! Would also like ask you what Polars version you are running.