The new implementation of a system to mine social media documents for evidence of foodborne illness outbreaks in NYC restaurants.
This software will be used by the NYC Department of Health and Mental Hygiene (DOHMH) to inform their practices.
Currently it's just research code.
This install guide is currently just intended for Mac OSX in the Terminal. Note that < >
encloses things you will need to fill in yourself
-
Make sure you have XCode Commandline tools installed, test this with
gcc
ormake
. If you get a prompt, follow it. -
If you don't have it, install Homebrew:
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-
Properly install python with
brew install python
-
Now you have
pip
. Next install virtualenv and virtualenvwrapper withpip install virtualenvwrapper
-
Add the following lines to your
~/.bash_profile
:# virtualenv and virtualenvwrapper settings export WORKON_HOME=$HOME/.virtualenvs source </path/to/wherever/pip/installed/virtualenvwrapper.sh> # eg, /Users/thomaseffland/Library/Python/2.7/bin/virtualenvwrapper.sh
Make sure to source this file with
source ~/.bash_profile
or by opening a new Terminal window -
Create a virtualenv to encapsulate the project with:
cd <~/mydevdir/FoodborneNYC> mkvirtualenv fbnyc
Note that in the future you'll need to activate the environment with
workon fbnyc
or deactivate it withdeactivate fbnyc
You'll know that you're in a virtual environment because your command prompt will be prefixed with the environment name, eg.:
(fbnyc) FoodborneNYC thomaseffland$
-
Install
git
if you don't have it already withbrew install git
-
Clone the repository from github with
git clone https://github.com/teffland/FoodborneNYC.git
-
Install all of the python dependencies
- If you just want to run the tool as is then use
pip install --upgrade -r deploy_requirements.txt
- If you want to develop the code, use
pip install --upgrade -r requirements.txt
- You may need to install additional software for the dev requirements, like a Fortran compiler.
- If you just want to run the tool as is then use
-
Congratulations! You now have FoodborneNYC installed. You can start messing with the code. Check out the command line options with
python main.py --help
-
To start using the data collection sources initialize your database schema with
python main.py initdb
NOTE: Due to the volume of data (even just from Yelp), it is highly recommended that you use a real DBMS instead of the preconfigured sqlite. SQlite is an in memory database and you are likely to have performance issues. Currently the system has only been tested for MSSQL Server, but SQLAlchemy does support many backends. For further info look here.
- When commiting, if you want to skip a CI build (eg, if you're just updating a README), make sure to add "[ci skip]" somewhere in your commit message.
-
Follow PEP #0008 for style and documentation guidance
-
When developing features, we will use a subset of the Github Flow. It's lightweight and conceptually simple but will encapsulate all of your work.
- We will maintain two main branches: production and development
- Production (master) will only be for stable releases
- Development (dev) will be the main working branch (think of this as master for dev purposes)
- Everytime you want to build a feature, create a new branch from dev
- Name your branch explicitly, eg "tom-ui-refactoring"
- Commit to your branch often
- Push your local commits to the central repo branch whenever you finish a work-session
- When you're ready to merge your feature back into dev, issue a pull request
- We will review your changes and make sure it passes all testing
- When we are ready to productionalize a version of the dev branch, it'll get merged with master
-
I highly suggest using git-flow to make using this workflow super convenient
-
We'll use TravisCI for continuous integration on pull requests. This saves us a lot of work. See this for an explanation
-
Write your unit tests with
pytests
-
Use
pylint
for some helpful automated code review ** TODO (teffland) ** - add testing coverage metrics -
Eventually, we'll use Sphinx and ReadTheDocs for automated documentation
- A useful reference for many things python. Specifically on how to write "pythonic" (idiomatic python) code
- A great resource on python best practices continually updated by python experts: Hitchhiker's Guide to Python