feature request: strategy to read output from a log file instead of stdout and stderr

[patricorgi]

Did you check existing requests?

• I have searched the existing issues

Describe the feature

Allow overseer to take command out put from a log file instead of stdout and stderr. Provide timeout for log file to be created.

Provide background

I work with Makefile and run lots of make commands. Usually I submit these commands to a different cluster using HTCondor. The cluster shares the same disk drive with my work node, but it has more CPU cores and much larger RAM.

It can run much faster than my work node but the problem is that the output does not display on my work node as it’s being executed on a different machine. So I have to rely on monitoring a log file of the submitted job. This can be done with less +F output.log.

I’m wondering how I can seamlessly achieve this workflow with overseer. I want to be able to

1. Still be able to get the list of make options with OverseerRun
2. Instead of using the default terminal strategy, submit the make command to the cluster which means create a job submission file and use condor_submit to submit the job
3. It usually takes a while for the cluster to start running the job, so a certain timeout option must be set to check if the output.log file is created.
4. Display the output by reading the output.log
5. Parse the output into quickfix list

What is the significance of this feature?

strongly desired

Additional details

No response

[Steven Arcangeli]

This is a cool use case! It should be possible by writing a custom strategy. The type definition of a strategy is here:

overseer.nvim/lua/overseer/strategy/init.lua

Lines 6 to 14 in fdcd46c

 ---@class overseer.Strategy 
 ---@field name string 
 ---@field new function 
 ---@field reset fun(self: overseer.Strategy) 
 ---@field get_bufnr fun(): number|nil 
 ---@field start fun(self: overseer.Strategy, task: overseer.Task) 
 ---@field stop fun(self: overseer.Strategy) 
 ---@field dispose fun(self: overseer.Strategy) 
 ---@field render nil|fun(self: overseer.Strategy, lines: string[], highlights: table, detail: number) 

You can browse that directory to see other examples of strategies and how they are defined. In particular, take a look at the test strategy that we use for unit tests. It is very minimal, so it's easy to see exactly what the responsibilities of the strategy are.

You can put your custom strategy in <runtimepath>/lua/overseer/strategy/htcondor.lua, or if this is something you think other could benefit from feel free to submit a PR!

One additional note: you shouldn't have to do anything for step 1. The Makefile targets will continue to be parsed correctly and fed through the system to end up at your custom strategy. Likewise, step 5 is also taken care of for you. As long as your strategy produces the output, the components that parse output and populate the quickfix should work the same.