Auto-cleaning of Midway logs

[Patrick de Perio]

We regularly reach file quota on /project, likely due to continuous production of job logs. I suggest to write a simple function in the job script that removes logs older than some threshold.

[Francesco Lombardi]

Ciao Patrick

There are two options maybe:

1. don't touch the directories of the present pax version, but delete together with the PurgeProcessed also the old log directories. We should find a way to identify automatically the directory where cax put the log, maybe in the config file.

2. crate check in the bash script on the number of log files present in the $JOB_WORKING_DIR. If they are more than 10 can be deleted. But anyway when we purge the processed file we will remove also the logs.

I will try to implement the first on before

[Patrick de Perio]

Ciao Francesco,

1. Right now, the path is hardcoded and built here. I guess you can hardcode in PurgeProcessed too, since we don't expect this to change often.

2. Since we don't frequently run PurgeProcessed, this would be preferable for continuously cleaning. As you mentioned, you can add a check here.

Thank you