sanger / labwhere

A tool for tracking uniquely barcoded labware

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

GPL-756 Tweak Event validation

KatyTaylor opened this issue · comments

User story
As someone using the Event Warehouse for reporting, I want the LabWhere events in there to be as reliable as possible. Currently, LabWhere fails to fire events for Audits that do not reference a valid location or labware. This is unnecessary validation as the location or labware could have been deleted since the event happened, and the audit record contains the location and labware barcode so we can just use that. The events are meant to just replicate the audit record, they don't need to do extra validation.

Acceptance criteria
To be considered successful the solution must allow:

  • Audits can be fired as Events even if the Location or Labware they reference does not exist
  • What if the Location or Labware barcode is missing?

Tasks

  • Code
  • Data - re-fire events

Additional context
This came out of #243

Data fix - the plan is to catch a lot of the events that failed validation when I did this migration - #243

So I will re-fire Events for the same set of Audits - all Labware Audits up to and including Nov 25th.
For most of them, re-firing won't change anything as they will already be in the Warehouse. (** test this on one or two first**)
But for those that failed validation previously, hopefully some or all will now go into the warehouse.

SQL to check how many relevant events before that date:

SELECT Count(e.id)
FROM events e
JOIN event_types et ON e.event_type_id = et.id
WHERE lims_id = 'LABWHERE'
  AND et.key IN ('labwhere_create', 'labwhere_update', 'labwhere_uploaded_from_manifest', 'labwhere_update_when_location_emptied')
  AND e.occured_at < '2020-11-26 00:00:00';

Code to run:
bundle exec rake labware_events:create END_DATE=2020-11-26

Deployed and re-fired events as in previous comment.
Before migration, SQL of all LabWhere events before 2020-11-26 counted 564,886
After migration, same SQL counted 595,129

Therefore 30,243 new Events were fired.

Searching in Kibana for messages including "LABWHERE EVENTS: Event could not be created for audit with id=" gave 85 records.
That's the same number as had a blank Auditable id, from my analysis on #243, so I think it's right that those should fail to fire Events.

Closing this issue now.