Giters

monk-ee / SplunkAppforAWSBilling

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

UsageStartDate 'fudge' misfiring

reillysg opened this issue · comments

commented

We've had two issues with this otherwise brilliant app

  1. By default - when we run the ProcessOlderReports.py all the timestamps generated come up identically as midnight on the day of the file being. This seems to be because of the fudge on line 206 or ProcessOlderReports.py - when I remove that check and call to fudge the data ingests and timestamps OK. Not good enough at python to figure out why the check fails when the UsageStartDate field really is there.

  2. I have to run the ProcessOlderReports.py manually as installing the app into splunk doesn't seem to install any kind of cron to run the actual ingester. Looking at ProcessDetailedReport.py, which looks the most likely candidate for cron - I cannot see it ever actually calling into splunk to insert data.

Any advice welcome

commented
  1. I need to do this better - open to suggestions? The line item appears with a start date but no end date.

It might make more sense to either:

  • divide it by 24 and set every hour the divided value across the day
  • set it to an arbitrary time like 9am on the same day
  1. Yeah, this was a quick fix for another request. The right way to do this is to look back in the past for the last 12 months reports (if they haven't been processed) and just process them. This could be then added to the scheduler to run all the time. I think I will make the time it goes back configurable so it's up to you how far back in the past it will collect logs for.
commented

It seems that AWS does not have blank line items in their report anymore. This is good, fudge date is not necessary so I will take it out and log the misfires.