from param

[Michael Gorsuch]

I need to be able to query a different start time than 60s ago. I should be able to do:

curl https://api.canary.io/checks/27b185a1-313f-4585-92b5-507f94b96075/measurements?from=[some_epoch]

[Michael Gorsuch]

cc canaryio/meta#9 in which @jagedthedrummer suggests something very similar, but using the param since.

[Jeremy Green]

After looking at this a little bit last night I think this is more complex than I had realized. Here's the scenario that I'm using to think through it.

Assume for this discussion that we have sensord running in NY and HKG, and only one instance of canaryd running in NY. At time X both instances of sensord take a measurement and send it over the wire to canaryd (both with t=X). At time X+1 canaryd receives the measurement from NY. At time X+2 sensord receives a request for the list of measurements, so it sends the one measurement that it has from NY. At time X+3 the measurement from HKG is received by canaryd.

Now if a request were to come in from the same client for measurements with from=X+2, what happens? We need to return the HKG measurement, but if we rely on t (as reported by sensord) then it will not be sent.

This led me to think that canaryd should add a server side "received" timestamp to each measurement as they come in, and also include a server side timestamp in each measurements response. In this scenario the first response from canaryd would be something like:

{
  canaryTime : Y,
  measurements : [
    { (NY measurement data for time X) }
  ]
}

Then the client could send a request with from=Y, and the response would be something like:

{
  canaryTime : Y+1,
  measurements : [
    { (HKG measurement data for time X) }
  ]
}

I think it would be relatively straight forward to make this happen within a single canaryd instance, but it seems like it could get very messy with multiple instances of canaryd scattered around the globe.

Thoughts?

[Michael Gorsuch]

This is good stuff and a valid concern - thank you. You are absolutely right that in the current model, you'd be unknowingly dropping things on the floor and the client view of the world would never be repaired.

I'm thinking that the simpler way for a client to get a near-synchronous view of things is going to be via streaming endpoints on canaryd - either through websockets or chunked http (I'd like to offer both). Events can come out of order if need be, etc, but as long as the client was connected, events would be received.

If a client were to become disconnected, it could simultaneously resubscribe to the stream and poll the existing GET /checks/:id/measurements?range=N to fill in the gaps. Since all measurements have a unique ID, dedupe would be possible client side.

[Jeremy Green]

👍

[Michael Gorsuch]

Closing in favor of #54. Seems to be a step in the right direction.