Add a driver for processing and working with spark dataframes
skanjila opened this issue · comments
The goal is to work with really large datasets and extract the results of large queries into a spark dataframe, this will allow us to work with pqs and spark to do large scale feature transformation and processing, we'll need to do a little design around this before implementation