esss_fix_format

Simple code formatter and pre-commit checker used internally by ESSS.

Imports sorted using isort
Trim right spaces
Expand tabs
Formats Python code using PyDev Code Formatter or black
Formats C++ code using clang-format if a .clang-format file is available

Install

conda install esss_fix_format

Note:

If executed from the root environment (or another environment) isort will classify modules incorrectly, so you should install and run it from the same environment you're using for your project.

Usage

Use fix-format (or ff for short) to reorder imports and format source code automatically.

To format files and/or directories:
```
fix-format <file1> <dir1> ...
```
Format only modified files in Git:
```
fix-format --commit
```
Or more succinctly:
```
ff -c
```

Options

Options for fix-format are defined in the section [tool.esss_fix_format]] of a pyproject.toml file. The TOML file should be placed in an ancestor directory of the filenames passed on the command-line.

Exclude

A list of file name patterns to be excluded from the formatting. Patterns are matched using python fnmatch:

[tool.esss_fix_format]
exclude = [
    "src/generated/*.py",
    "tmp/*",
]

Black

Since version 2.0.0 it is possible to use black as the code formatter for Python code.

fix-format will use black automatically if it finds a [tool.black] section declared in pyproject.toml file.

See "Converting master to black" below for details.

Migrating a project to use fix-format

Follow this steps to re format an entire project and start using the pre-commit hook:

You should have ff available in your environment already:

$ ff --help
Usage: ff-script.py [OPTIONS] [FILES_OR_DIRECTORIES]...

  Fixes and checks formatting according to ESSS standards.

Options:
  -k, --check   Check if files are correctly formatted.
  --stdin       Read filenames from stdin (1 per line).
  -c, --commit  Use modified files from git.
  --git-hooks   Add git pre-commit hooks to the repo in the current dir.
  --help        Show this message and exit.

For each file you don't want imports reordered add isort:skipfile to the docstring:
```
"""
isort:skip_file
"""
```
Commit using -n to skip the current hook.
If there are any sensitive imports in your code which you wouldn't like to ff to touch, use a comment to prevent isort from touching it:
```
ConfigurePyroSettings()  # must be called before importing Pyro4
import Pyro4  # isort:skip
```
If you want to use clang-format to format C++ code, you should copy the .clang-format file from esss-fix-format to the root of your project. This is optional for now in order to allow incremental changes (if this file is not present, the legacy C++ formatter will be used):
```
$ cd /path/to/repo/root
$ curl -O https://raw.githubusercontent.com/ESSS/esss_fix_format/master/.clang-format
```
If you want to use black to format Python code, add a pyproject.toml to the root of your repository; an example can be found in "Converting master to black" below.
Activate your project environment:
```
$ conda activate myproject-py36
```
Execute:
```
$ cd /path/to/repo/root
$ ff .
```
After it completes, make sure there are no problems with the files:
```
$ ff . --check
```
Note

if the check fails, try running it again; there's a rare bug in isort that might require to run ff /path/to/repo/root twice.

Commit:
```
$ git commit -anm "Apply fix-format on all files" --author="fix-format <fix-format@esss.com.br>"
```
Push and run your branch on CI.
If all goes well, it's possible to install pre-commit hooks by using ff --git-hooks so that any commit will be checked locally before commiting.
Profit! 💰

Migrating from PyDev formatter to black

Migrating an existing code base from a formatter to another can be a bit of pain. This steps will help you diminish that pain as much as possible.

Converting `master` to black

The first step is converting your master branch to black.

Add a pyproject.toml project with this contents:

[tool.black]
line-length = 100
skip-string-normalization = true

Your root directory should have a .isort.cfg file with the same contents as one in the root of this repository, add those lines:
```
[settings]
profile=black
no_sections=True
force_alphabetical_sort=True
```
This will use black-like grouping, and clump imports together regardless if they are standard library, third party, or local. This avoids getting different results if you have a different environment activated, or commiting from an IDE.

Commit, and save the commit hash, possible in a task that you created for this conversion:
```
$ git commit -anm "Add configuration files for black"
```

Execute on the root of the repository:
```
$ fix-format .
```
Ensure everything is fine:
```
$ fix-format --check .
```
If you don't see any "reformatting" messages, it means everything is formatted correctly.

Commit and then open a PR:

$ git commit -anm "Convert source files to black" --author="fix-format <fix-format@esss.com.br>"

Porting an existing branch to black

Here we are in the situation where the master is already blacken, and you want to update your branch. There are two ways, and which way generates less conflicts really depends on the contents of the source branch.

merge -> Fix format

Merge with the target branch, resolve any conflicts and then commit normally.
Execute fix-format in the root of your repository:
```
$ fix-format .
```
This should only change the files you have touched in your branch.

Commit and push:

$ git commit -anm "Convert source files to black" --author="fix-format <fix-format@esss.com.br>"

Fix format -> merge

Cherry-pick the commit you saved earlier on top of your branch.
Execute fix-format in the root of your repository:
```
$ fix-format .
```
(In very large repositories, this will be a problem on Windows because of the command-line size, do it in chunks).

Fix any conflicts and then commit:

$ git commit -anm "Convert source files to black" --author="fix-format <fix-format@esss.com.br>"

Developing

Create a conda environment (using Python 3 here) and install it in development mode.

Make sure you have conda configured to use ``conda-forge`` and ``esss`` conda channels.

$ conda install -n base conda-devenv
$ conda devenv
$ source activate esss-fix-format-py36
$ fix-format --git-hooks
$ pytest

When implementing changes, please do it in a separate branch and open a PR.

Releasing

The release is done internally at ESSS using our conda-recipes repository.

License

Licensed under the MIT license.

volpatto / esss_fix_format