hangtime79 / dss-plugin-sql-code-extractor

Generate a text report with the SQL from all visual and code recipes from a given project.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

SQL Code Extractor

Overview

  • One of the strengths of DSS is its ability to run visual recipes using compute infrastructures (SQL databases, Hadoop clusters, etc). See: https://www.dataiku.com/learn/guide/getting-started/dss-concepts/where-does-it-all-happen.html

  • To that end, DSS translates visual recipes into code that can then be pushed down to the data source, whether traditional SQL for relational databases (Postgres, MySQL , Oracle etc.) or other SQL-like dialects for Hadoop clusters (Hive, Impala, SparkSQL). DSS users can also directly write code recipes in those languages.

  • The purpose of this macro is to build a report consisting of SQL code from these recipes (visual and code). The macro can build a report for a single project or for all projects in an instance.

Usage

Input

  • This macro takes as input an existing folder in the project where the macro is used.
  • The user can choose to either run the macro on the current project or across all the project they have access to on the instance.

Output

  • The output is a zip file dumped in the chosen folder and a report table that shows the count of recipes per project from which SQL code was retrieved.

  • The structure in the zipped folder is sql_extract-{extractionTime}/{projectKey}/{recipeName.sql}

Limitations

  • This macro is unable to retrieve SQL from the pivot recipes.

License

Apache Software License

About

Generate a text report with the SQL from all visual and code recipes from a given project.


Languages

Language:Python 100.0%