Improve source controller sync processing

[Paul Carlton]

At present the mapping function that processes the gitrepository.source.toolkit.fluxcd.io custom resources gets the current repository contents from the source controller, identifies the addon layers that use it and creates a link to the addon layer's data directory then requeues the addon layers for processing.

This works but has some issues. The process of getting the repository data from the source controller and unpacking it into the local filesystem involves deleting the existing copy of the data to avoid deleted files being retained. This creates a window when the symbolic link used by the addon layers to access this data points to a non existent directory. This causes the addon layer to be marked as failed. The subsequent reprocessing of the addon layer will find the data and continue but this generates errors and status updates that are alarming for the user.

The repos package that is used to process the gitrepository.source.toolkit.fluxcd.io custom resources contains a Repos object that keeps track of repos. This contains entries for each revision of a repo. A better solution would be for the Repos to contain Repo objects that relate to gitrepository.source.toolkit.fluxcd.io custom resources and introduce a 'RepoRevision' object that relates to a particular git commit in the repo.

I also propose that the mapping function should just manage the Repos, adding, deleting Repo objects to the map in Repos and requeuing any addon layers that use that Repo.

We should add code to the addon layers reconciliation processing to look for the Repo they use in Repos and then look in the Repo for a RepoRevision matching the Repo's current revision. If this is not present it will be created and data retrieved from the source controller. Then the symbolic link for the addon layers data can be updated to point at the new revision or adjusted if need be to point at pre-existing directory, which will occur if multiple addon layers are using the same gitrepository.source.toolkit.fluxcd.io custom resource and one has created and updated a new revision.