Logging is prepending a timestamp on our JSON output
bmenrigh opened this issue · comments
Example:
2017/03/02 22:27:30 {"timestamp":"2017-03-02 22:27:30.00813237 +0000 UTC","bytes_client":"144","http_method":"GET","url_path":"/test/path/here.txt?crap","http_version":"HTTP/1.1","http_user_agent":"curl/7.52.1","dest_name":"127.0.0.1:3333","http_referer":"\u003cscript\u003ealert(\"pwned\");\u003c/script\u003e","src_ip":{"IP":"127.0.0.1","Port":58328,"Zone":""},"dest_ip":"127.0.0.1:3333","raw_data":"474554202f746573742f706174682f686572652e7478743f6372617020485454502f312e310d0a486f73743a203132372e302e302e313a333333330d0a557365722d4167656e743a206375726c2f372e35322e310d0a4163636570743a202a2f2a0d0a526566657265723a203c7363726970743e616c657274282270776e656422293b3c2f7363726970743e0d0a0d0a"}
And:
2017/03/03 22:29:18 {"raw_data":"476554202f746869735f4745545f7761735f47655420485454502f312e300d0a0d0a"}
Maybe we should just roll our own logging library? That would eliminate the dependency on an external lib like Lumberjack.
So I have done a bunch of reading on this and I think we can do this two ways
-
We write our own logger that when it reaches 10 minutes it Closes the file, renames, and then opens a new file. Then resumes logging.
-
We use linux os level logrotate, the issue with this is that "Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost." We would have to have the script looking for a signal from logrotate to stop writing to the file handle and then resume after rotation is complete. Then we just have a cronjob that runs logrotate every 10 minutes.
Thoughts?
If we use an OS-level log rotation tool we still have to signal the sinkhole to flush and close the file and then signal it again when it should open a new log. That's messy. The primary feature of this tool is logging so we should put effort into making sure the logging is done right.
I don't think we need to do any renaming at all. Let's just use the build the timestamp into the file name from the start. Something like netsarlacc-201703292100.log and then the next ones would be netsarlacc-201703292110.log, netsarlacc-201703292120.log, etc.
I suppose we could start by opening netsarlacc-201703292100.tmp and then when we close to open another, we move it to netsarlacc-201703292100.log.
Opening and closing files is simple. I don't see why we wouldn't do it in our own code.
This has been fixed with the new logging code.