Multithreaded
calumrussell opened this issue · comments
To improve performance further alator should offer the option of multi-threaded components.
This is a complex improvement because of the risk of lookahead bias in backtests. Other similar libraries have multi-threaded components but they achieve this by allowing trading strategies to trade instantaneously at the market price.
This isn't only an issue with high-frequency strategies but end-of-day strategies will overestimate out-of-sample performance if the strategy is allowed to trade instantaneously on close rather than on tomorrow's open, which would be the next possible price to trade at live.
The way to remove lookahead bias in multi-threaded backtesting is to have strategies that can run async but block the trading loop until they are finished creating orders, and then run the rest of the loop synchronously. In this case, I think that the broker and exchange would need new thread-safe structures (Arc<Mutex>).
I believe it would also be possible for the exchange to be multi-threaded too (but haven't tested this fully). In the current implementation, the broker does not wait for trade confirmations anyway because the exchange only executes orders on the next tick (this is why we have the complex Ready/Waiting state controls) so running this async would make sense.
However, this would seem to introduce the possibility of a race condition when an order is passed and executed but the broker state isn't updated, and the strategy keeps issuing orders. If we have a sync::mspc::channel then I am not clear on where in the loop we block to receive messages and update state.
First stage is to implement multi-threaded trading strategies and thread-safe data structures where necessary.
@calumrussell https://docs.rs/rayon/latest/rayon/ Could be an easy dropin to add multi-threaded logic for stuff where looping is used, as an easy win. It shouldn't add lookahead bias if we are picky with the places we add it. If that sounds interesting, I can add it and come up with a draft PR. :)
@calumrussell https://docs.rs/rayon/latest/rayon/ Could be an easy dropin to add multi-threaded logic for stuff where looping is used, as an easy win. It shouldn't add lookahead bias if we are picky with the places we add it. If that sounds interesting, I can add it and come up with a draft PR. :)
All I have at this stage is a toy example of the control flow: the exchange blocks until the traders complete then ticks forward before executing orders, and then blocks until traders complete, etc.
I am not 100% clear on how this should all be controlled and synchronized. Right now, everything is triggered within the loop in SimContext so that is the logical place for control. But I am also considering adding an Event log, which is a big omission currently, which would need to be synchronized as well.
So I would like to do this stage myself, as I want to be able to understand exactly how this part works. There will likely be a lot more work required on this after this point.
The problem with rayon, as I understand it, is that it is suited for divide-and-conquer problems. I think this would work with vectorized backtests but there are dependencies between time steps for event-driven backtests. So it isn't like sorting where you can split the task between cores, and then join the results together. You only the result for tick 9 after tick 8. I am planning on using tokio/futures, the main differentiating factor between libraries is the scope of channels (i.e. whether mspc or broadcast...or even whether channels are needed if there is no Event log).
Closed with - #34
Unsurprisingly required a substantial rewrite but has basic feature parity with the option to run backtest with sync/async components. Communication between threads using channels. Biggest architectural change is moving price state within exchange (both within sync/async impls), this will require further changes to DataSource in order to consolidate but does mean that existing components could be used in live trading environment.
Docs still need to be rewritten but the core changes are complete.