Support for loading data from stages

[Florian Kühnlenz]

Hi,

Would it be possible to support loading of data from a stage? (https://docs.snowflake.com/en/user-guide/data-load-considerations-load)

I am not sure from the docs if this would be even inside the scope of this project.

However it would help us a lot to test our complete ETL workflow.

[Oliver Mannion]

Hi @tharwan this might be possible ... could you share example SQL statements you'd like support for, both creating the stage and loading from it?

[Florian Kühnlenz]

Loading from a stage looks something like this:

COPY INTO my_table
FROM @my_stage/my_file
FILE_FORMAT = (FIELD_DELIMITER = ';')

Creating a stage is not so relevant in our case but looks like this:

CREATE STAGE my_ext_stage
URL='azure://myaccount.blob.core.windows.net/load/files/'
STORAGE_INTEGRATION = myint;

maybe related: tobymao/sqlglot#2463

[Oliver Mannion]

Thanks! From the above I gather you want support for CSV files.

I think creating the stage would be needed for fakesnow to know where to find the files to load.

[Florian Kühnlenz]

another option could be to just look locally for a file and ignore the @my_stage part.

e.g.

COPY INTO my_table
FROM @my_stage/my_file
FILE_FORMAT = (FIELD_DELIMITER = ';')

would translate to

COPY my_table
FROM my_file
(FORMAT CSV, DELIMITER ';');