pull, add-environment: if rewriting .skeema file, preserve comments/format

[Alexander Korsunsky]

When running skeema pull, the file .skeema is always read and re-generated.

I believe that this is conceptionally wrong, if the .skeema file is considered an "option file". Option files and settings should always be considered read-only by programs, and never touched or modified. The only time where it makes sense for the option file to be modified is by operations that have it as their explicit purpose, for example skeema init or skeema add-environment.

Practically, this is an issue in the following cases:

• It is impossible to add comments in order to clarify or explain the choice for certain options. Adding comments to the .skeema file is syntactically valid, but they are removed by every single invocation of skeema pull.
• If skeema directories and .skeema files are version-controlled (which is one of the main advantages of the skeema tool), then people with different versions of skeema will cause a change in the generator= line of the .skeema file every time they run skeema pull, even if there would be no changes otherwise. This can lead to avoidable merge conflicts.

Would it be possible to change skeema to do one of the following?

• a) Simply not touch the file at all and open it read-only except when skeema init or skeema add-environment are used
• b) Not touch the file if no changes to it are required
• c) Add an option that prevents any changes to files at all, or an option that prevents changes when an operation does not require them

[Evan Elias]

Sorry but Skeema makes no guarantees that it doesn't touch .skeema files, nor that these files are treated as "read only" in any way. In fact quite the opposite; the documentation on the relevant options clearly states that skeema pull will update these options when needed. This has always been the case.

When running skeema pull, the file .skeema is always read and re-generated.

It is always read, but your assertion that it is always re-generated is not correct. It is only rewritten when it needs to update an option -- your request (b) above is already how it works. The .skeema file is rewritten only when one of these options needs to be updated:

• flavor: for example if you've upgraded from MySQL 5.7 to MySQL 8. Relevant function is updateFlavor().
• generator: for example if you've upgraded Skeema, or switched from Community to Premium edition. Relevant function is updateGenerator().
• default-character-set and default-collation: for example if an ALTER DATABASE was run out-of-band, or if your environments are unexpectedly using different database-level charset/collation for some reason. Relevant function is updateCharSetCollation().

I agree that it is annoying to lose formatting or comments when this happens. But generally it's easy enough to just git checkout -- .skeema and then manually edit back in whatever automatic change skeema pull is trying to make.

One other solution is for you to make .skeema read-only and/or immutable at the filesystem level. For example on Linux, chattr +i .skeema will make the file immutable. If skeema pull cannot rewrite .skeema, it ignores this failure entirely for generator, and only logs a warning but otherwise ignores it for flavor. The only fatal case is if skeema pull fails to rewrite .skeema due to a default-character-set or default-collation divergence, since this is an important discrepancy which potentially affects interpretation of DDL.

At some point in the future, the .skeema handling logic will be updated to preserve comments and formatting, but it hasn't been high priority relative to other feature improvements.

Regarding generator causing unwanted changes: best practice is to run the same version and edition of Skeema throughout your company, otherwise you run the risk of different behaviors, options in .skeema files not existing in older versions, etc. The premium edition adds skeema update to make updating a trivial operation.

[Evan Elias]

Converting this to an issue for tracking preserving comments and formatting in .skeema files, since that would greatly improve the core complaint if I understand correctly.