Maintenance for long running clusters
harshit2283 opened this issue · comments
Harshit commented
Currently Arroyo stores all metadata for checkpoints in checkpoints
table, overtime it grows upto a significant size which may impact performance.
Immediate solve (by @mwylde ) is to run this query manually ->
DELETE FROM checkpoints
WHERE checkpoints.id != (
SELECT id FROM checkpoints
WHERE job_id = '{{ JOB_ID }}'
ORDER BY finish_time DESC
LIMIT 1
) AND job_id='{{ JOB_ID }}';
Jackson Newhouse commented
Just merged the fix