Databox logger
A web-based interface to collate audit information from Databox components such as stores. Data is collected using the Zest protocol and stored to a time series database.
Getting started
starting a store
We will start zestdb with logging enabled to see the interaction.
docker run -p 5555:5555 -p 5556:5556 -it --name zest --rm jptmoore/zestdb /app/zest/server.exe --secret-key-file example-server-key --enable-logging
starting the logger
We will start the web server on its default port of 8000 without TLS enabled.
docker run -it --network host -it --name logger --rm jptmoore/zestlogger /home/databox/logger
request audit information
Logging takes place through Zest observations. To setup an observation we need to make a PUT request with some JSON. The length of an observation is determined by the max-age field value specified in seconds. A zero value means the observation will not timeout. The PUT request will return immediately so you should monitor the console for issues. The PUT request requires the id of the datasource you want to observe supplied in the path, for example, /observe/foo.
docker run --network host --rm byrnedo/alpine-curl -X PUT 127.0.0.1:8000/observe/foo --data '[{"path": "/ts/foo"}, {"key": "vl6wu0A@XP?}Or/&BR#LSxn>A+}L)p44/W[wXL3<"}, {"main_endpoint": "tcp://127.0.0.1:5555"}, {"router_endpoint": "tcp://127.0.0.1:5556"}, {"max_age": 60}, {"token": ""}]'
Interact with store
Once we have setup some observations we can post to the store to generate some log entries. For example, below we post 5 times to the store.
docker run --network host -it jptmoore/zestdb /app/zest/client.exe --server-key 'vl6wu0A@XP?}Or/&BR#LSxn>A+}L)p44/W[wXL3<' --path '/ts/foo' --mode post --payload '{"value": 42}' --loop 5
read console log
The log has the format: timestamp, server-name, client-name, method, uri-path, response-code
Starting observation to foo
1551272344801 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65
1551272345810 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65
1551272346817 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65
1551272347825 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65
1551272348830 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65
read from database
We can use the time series API to read the stored logs.
docker run --network host --rm byrnedo/alpine-curl localhost:8000/ts/foo/latest
produces:
[{"timestamp":1551272348812208,"data":{"value":"1551272348830 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65"}}]
combining logs
We can combine logs by comma-separating the the id of each data source when using an API call.
Let's add data to another time series first.
docker run --network host -it jptmoore/zestdb /app/zest/client.exe --server-key 'vl6wu0A@XP?}Or/&BR#LSxn>A+}L)p44/W[wXL3<' --path '/ts/bar' --mode post --payload '{"value": 42}' --loop 5
We can now retrieve data from both time series. Note, this request is equivalent to calling /ts/foo,bar/earliest.
docker run --network host --rm byrnedo/alpine-curl localhost:8000/ts/foo,bar/first/1
produces:
[{"timestamp":1551272344787363,"data":{"value":"1551272344801 c2480de2b3e0 linuxkit-025000000001 POST /ts/foo 65"}},{"timestamp":1551364092420804,"data":{"value":"1551364092431 b160be0536a4 linuxkit-025000000001 POST /ts/bar 65"}}]
Time series API
Read latest entry
URL: /ts/<id>/latest
Method: GET
Parameters: replace <id> with an identifier
Notes: return the latest entry
Read last number of entries
URL: /ts/<id>/last/<n>
Method: GET
Parameters: replace <id> with an identifier, replace <n> with the number of entries
Notes: return the number of entries requested
Read earliest entry
URL: /ts/<id>/earliest
Method: GET
Parameters: replace <id> with an identifier
Notes: return the first entry
Read first number of entries
URL: /ts/<id>/first/<n>
Method: GET
Parameters: replace <id> with an identifier, replace <n> with the number of entries
Notes: return the number of entries requested
Read all entries since a time (inclusive)
URL: /ts/<id>/since/<from>
Method: GET
Parameters: replace <id> with an identifier, replace <from> with epoch milliseconds
Notes: return entries from time provided
Read all entries in a time range (inclusive)
URL: /ts/<id>/range/<from>/<to>
Method: GET
Parameters: replace <id> with an identifier, replace <from> and <to> with epoch milliseconds
Notes: return entries in time range provided
Delete all entries since a time (inclusive)
URL: /ts/<id>/since/<from>
Method: DELETE
Parameters: replace <id> with an identifier, replace <from> with epoch milliseconds
Notes: deletes entries from time provided
Delete all entries in a time range (inclusive)
URL: /ts/<id>/range/<from>/<to>
Method: DELETE
Parameters: replace <id> with an identifier, replace <from> and <to> with epoch milliseconds
Notes: deletes entries in time range provided
Length of time series
URL: /ts/<id>/length
Method: GET
Parameters: replace <id> with an identifier
Notes: return the number of entries in the time series