Sicarator

Instant Setup & Best Quality for Data Projects!

Sicarator is a CLI generator for Data Science projects. It is built with Yeoman and maintained by Sicara.

Main components of the generated project

Python development environment with:
- Poetry for dependency management
- Pytest for testing
- Black for code formatting
- Ruff for static analysis
- Mypy for type checking
- Pre-commit for git hooks
Continuous Integration of your choice:
(optional) API:
- built with FastAPI
- containerized with Docker
- deployed on AWS (API Gateway, ASG, ECS, EC2)
- provisioned with Terraform
- testable with a Postman collection
(optional) Data Versioning and Pipelines with DVC
(optional) Data Visualization with Streamlit

Requirements

Pyenv

Install pyenv to manage your Python versions and virtual environments:

curl -sSL https://pyenv.run | bash

If you are on MacOS and experiencing errors on Python install with PyEnv, follow this comment

Poetry

Install Poetry to manage your dependencies and tooling configs:

curl -sSL https://install.python-poetry.org | python - --version 1.5.1

If you have not previously installed any Python version, you may need to install one and set it as your default Python version before installing Poetry:

pyenv install 3.10.6 # or any recent version
pyenv global 3.10.6

Node.js

Install a stable version of Node.js (v16.x.x) if you don't have one. The Sicarator is indeed a Yeoman generator, which is a Node.js module.

For linux:

curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs

Yeoman

Install Yeoman:

npm install -g yo

Usage

Before generating a new project, make sure to install last version of Sicarator by re-running:
```
npm install -g sicara/sicarator
```
Generate a new project with:
```
yo sicarator
```

Troubleshooting

I don't have the correct permissions to install yeoman

Example error:

npm ERR! code EACCES
npm ERR! syscall rename
npm ERR! path /usr/lib/node_modules/generator-sicarator
npm ERR! dest /usr/lib/node_modules/.generator-sicarator-cELCsz5l
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, rename '/usr/lib/node_modules/generator-sicarator' -> '/usr/lib/node_modules/.generator-sicarator-cELCsz5l'
...

Solution: give yourself the ownership of npm and node_modules :

sudo chown -R $(whoami) ~/.npm
sudo chown -R $(whoami) /usr/lib/node_modules # or /usr/local/lib/node_modules depending on where node modules are installed

I have the correct git credentials, but installation is throwing a git error

Example error:

git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Solution: you are probably using sudo to run the installation, which doesn't use the same git credentials. See solution above to install the project with your current user profile.

Contribute to the project

Install the project

Install the project with:

git clone git@github.com:sicara/sicarator.git

and install dependencies

npm install

To be able to run current dev version of the project, you can use the following command in your project directory:
```
npm link
```
- Next time you run yo sicarator, dev version will be used.
- If you want to go back to the production version you can re-run:
```
npm install -g sicara/sicarator
```

Dev tools

For debugging, you can run:

npx --node-options="--inspect" yo sicarator

To run the linter (eslint), you can use:
```
npm run lint
```
To run the tests, you can use:
```
npm test
```

patzhang / sicarator