Progress reporting introduces large performance hit
csparker247 opened this issue · comments
I recently discovered that one of my algorithms which I thought was long running (~7min), was actually quite fast when I disabled the progress reporting (15s!!!). This doesn't particularly surprise me since the indicators need to call std::flush to update the output stream, and doing so repeatedly with small intervals is not very efficient.
I wanted to open a discussion about if and how this might be mitigated in the library. I'm trying to think of solutions that don't involve threads, since that's not necessarily portable across platforms.
My initial idea would be to add an option UpdateType with values Eager or Interval. The former would update the stream as soon as progress is updated (the current behavior). The second would only update if a specific time interval (another option, UpdateInterval) has elapsed since the previous update. This should improve performance for all processes which tick faster than the specified interval. However, for processes which tick slower than the update interval, you'll probably end up with some sort of update aliasing, which I think would manifest as irregular updating? In that case, though, I don't know why you wouldn't just use Eager mode. Also, this probably would not be useful for the Indeterminate Progress Bar.
And in case this needs to be said, if this or some other idea seem reasonable and within scope, I'd be happy to contribute them.
Not incurring a big performance hit to report progress is a definite requirement (for me at least). I do not think you even need to have an UpdateType option. I think we could get by with a single option UpdateInterval of type chrono::duration . When it is 0, then you update all the time. When it is non-zero, then you use an additional data member, last_update_time and chrono::steady_clock::now() to see if it is time to update.
Have you experimented with any changes to reduce the performance hit yet? I am not that familiar with the indicators implementation, so I would be glad to work with you on this. As I said, I cannot use a progress reporting mechanism that is going to incur a significant performance penalty or block a thread in any way. So I either need to do something to indicators or keep looking for a new solution.
As my algorithm was fast enough that I could just remove the progress usage in my codebase, this became low priority for me and I haven't tested anything, preferring to wait to get some sort of feedback on the interface design.
I like the chrono::duration idea. It simplifies the interface in an intuitive way. One thing I'm noting now to help me remember during implementation, the progress output should always get printed for the final tick/set_progress/mark_as_completed call.
I'm under a deadline for the next few weeks, so I probably won't get to look at this seriously for a little bit longer. If you wanted to take a stab at it, the code is pretty self explanatory. I'd probably start by adding a check to print_progress() in include/indicators/block_progress_bar.hpp. Once it works there, it's easy enough to port to the other appropriate indicators.
I was running into the same issue.
I have an indicator with max progress in the millions/billions. So I was printing the progress a lot even when there was not much visual change.
I have another idea to reduce the printing load that is not time based. Which should be faster since getting the time is a system call.
My solution aggregates progress in larger chunks and adds a progress granularity option. A granularity of 100 means printing every 1%, granularity of 1000 means printing every 0.1%.
When the max, min progress and granularity is set the indicator calculates the amount of progress that has to be made until it prints again.
When the progress is set it simply compares to the needed amount.
An implementation can be seen in this branch specifically this commit.
Weirdly, this issue just popped up for me again in my code within the last week. Guess it's a good time for people running long running tasks.
From a UX perspective, I personally think I'm more interested in having the bar update the eta after it reaches a specified printing interval rather than waiting for it to report on a percentage. Particularly with long running tasks, the user always thinks it's hung. Is it hung at 1% or just taking a long time to get to 2%? Updating something in the progress bar helps that. I haven't profiled it, but I also think getting the system time is probably marginal compared to the current time spent flushing the buffer.
Anyway, I also see the merit in your approach. Not sure how compatible the two ideas are given the current code, but seems at least possible.
In my case, I got the performance back by calling set_progress only when I needed to:
int current_pct = static_cast<float>(current_progress) / static_cast<float>(total) * 100;
if (current_pct > previous_pct){
progress_bar.set_progress(current_pct);
}
previous_pct = current_pct;
It would be nice if the framework could have similar logic internally.