add an option to re-run on config changes

[toejough]

for instance, if I use pylint as a checker, I want to re-run whenever my .pylintrc file changes.