Can this project support custom sorting？
donhui opened this issue · comments
Now the artifacts sort by path:
artifacts = sorted(artifacts, key=lambda x: x["path"])
I wonder if We can support custom sorting rules, such as:
artifacts = sorted(artifacts, key=lambda x: (x['path'], x['created']))
For example, the artifacts log4j-web sort by path as follows, 2.6.2 is at the end, but I want to let 2.6.2 at the first when I execute cleanup.
So I think sort by path and created will slove the problem.
Based on you description, I don't think this is related to artifactory-cleanup, but artifactory service itself.
@zhan9san
Of course, it can be said that there is a problem with the artifact service.
One one hand, we can wait for artifactory to solve this problem,
One the other hand, If artifactory-cleanup can support sort, it will slove the problem too.
@donhui hi! could you show CleanupPolicy or describe a little your case - what do you want to remove in above screenshot.
As I understood it - you're trying to use keep_latest_n_file_in_folder with n=7 and got wrong result?
Sure, we can add a rule that will change sorting of the results, but it looks like you need to add additional rules too for your case.
So it'll be something like this:
CleanupPolicy(
'Delete files older than 30 days',
rules.repo('reponame'),
rules.sort_by(['path', 'created']),
rules.delete_last_n_folders(n=7),
),
@allburov Your understanding above is completely correct, and the solution you provide is also what I want.
@donhui is it a proxy repository or you build your own log4j? Who put those artifact there?
Because I'm afraid that if it's a proxy - it doesn't mean that 2.6.2 was created earlier then 2.18.0. In this case we need to support some sort of "sort by semver", but it's too hard to add for all type of repositories - they all have different path patterns
Could you clone the repository and change the line manually and run cleanup in our environment - if it suites you - I'll add the suggested above rule
I have manually changed the code in my own environment and have run cleanup in my environment. log4j-web is just an example. We usually cleanup local repositories.
I submit this issue in the hope that this function will become universal.
Thanks.
@donhui I think the best way to apply specific logic for ordering - is to create your own Rule and return new (or just sorted) ArtifactList.
The examples are for 1.0.0 version (which is coming, as you know #63)
# myrule.py
from typing import List
from artifactory_cleanup import register
from artifactory_cleanup.rules import Rule, ArtifactsList
class OrderByPathAndCreated(Rule):
def filter(self, artifacts: ArtifactsList) -> ArtifactsList:
"""Reorder artifacts in the right way"""
artifacts = artifacts.sort(key=lambda x: (x['path'], x['created'])
return artifacts
# Register your rule in the system
register(OrderByPathAndCreated)artifactory-cleanup:
server: https://repo.example.com/artifactory
user: $ARTIFACTORY_USERNAME
password: $ARTIFACTORY_PASSWORD
policies:
- name: Use your own rules!
rules:
- rule: Repo
name: "repo-name-here"
- rule: OrderByPathAndCreated
# after that rule artifacts will be resorted by OrderByPathAndCreated
- rule: OtherRulesThatWillUseNewOrdering
and run it as artifactory-cleanup --load-rules=myrule.py
@donhui also I removed the part artifacts = sorted(artifacts, key=lambda x: (x['path'])) because some rules (like KeepLatestNFile) relies on certain ordering itself and sorts artifacts via AQL (we should sort it in memory to support JCR tho, I'll do it too) #59
artifactory-cleanup/artifactory_cleanup/rules/keep.py
Lines 101 to 103 in b35c591