TimelyDataflow / timely-dataflow

A modular implementation of timely dataflow in Rust

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Logging nice-to-haves

utaal opened this issue · comments

We are about to land a PR #92 that overhauls the logging infrastructure. This issue keeps track of desirable features that should be possible with this infrastructure, but aren't yet designed/implemented.

  • re-introduce the log crate
  • allow ad-hoc logging of user code (i.e. allow users of timely to produce logging streams of a type picked by them)
  • support log filtering, and selectively enabling log statements
  • make sure that log streams can be flushed when necessary
  • remove the time crate
  • determine what timestamp resolution scale is appropriate, consider using a synchronisation point between processes as the time base for the logs
  • let users toggle logging for certain dataflows
  • Allow log processing within the same timely dataflow instance without going through the network.
  • allow users toggle logging for certain timestamp in certain dataflow (for tracing, debugging, etc)