NOTE: We are using torch with Cuda in our notebook, so please make sure your environment has access to GPU !
- project.ipynb
- model.bin: see instructions below
- sample_data.csv : contains 100 sample data points.
the model.bin is too big to be uploaded on to github and may suffer from corruption when using git lfs, so please follow this instruction to download model.bin
- clone the repo to your local environment
- download the model.bin from our google drive
- put the model.bin into your cloned repo. It should be in the same directory as the Project.ipynb
Note: It is recommended to run this notebook in Google Colab to avoid any sort of dependency issues. In which case please place/upload all required files listed above in the same directory in your google drive.
- Run all the blocks under "Set Up", this includes installing the dependencies at the top. Installing the required dependencies can take around ~1.5 min.
- In "Load Data" section, IF you are using your LOCAL ENVIRONMENT: if you downloaded all the files in the same structure as it is in github, then you DO NOT need to update the file paths in the first block of "Load Data." Else, please update the paths according to the comments in the notebook. And skip the second block under "Load Data."
- In "Load Data" section, IF you are using GOOGLE COLAB: please run the mount to mount your Google Drive by running the second block under "Load Data". You will need to update your file paths accordingly.
- In "Load Data" section, Run all remaining blocks (skiping block 2 if you are using local environment instead of Colab)
- In "Load Model" section, run the 1 block that is in this section, this can take around ~1 min to load in the trained model data.
- SKIP "Modeling Training" and "Model Saving"
- In "Generation" section, in the first block, the first line is commented out. If you wish to use a input from the original dataset, then please uncomment that first line and comment out the second line.
- In "Generation" section, If you want to try our own input, comment out the first line and uncomment the second line, then in the second line, please enter a list of ingredients in the following format : Ingredients: 1. amount ingredient 2. amount ingredient ...
- In "Generation" section, after setting the input you want, run the first block.
- In "Generation" section, there are subsections for different temperature values, you can run a specific subsection or all of the subsections.
- In "Evaluation" section, if you wish to perfrom evaluation, in the first block, there is a temperature variable, it is set at 1.5 (optimal), you can change it if you wish. Note that it has to be a positive float. Them run all blocks under "Evaluation" section.
- recipe1m directory: Contains python code for gathering data from Recipes1M+ database and code from cleaning data
- Project.html: html output of model notebook that contains the output for every cell
- Project.ipynb: Final project notebook, contains dataloading, model training, response generation, and evalution
- comments_dfs.ipynb: Contains code preview reddit post comments
- reddit_comments.py: contains code to extract reddit posts and their comments
- reddit_pushshift_api.py: contains code to set up pushshift to pull data from reddit
- sample_data: contains 100 sample data points
- tables.sql: contains SQL code for generating our table
- visuals.py: contains code for word cloud visual
- model.bin: final GPT2 model data