How to use backtest warmup property correct?
Pirat83 opened this issue · comments
Hello,
I currently have some questions about the warmup property in the Strategy.backtest(...) method. My goal is to set the propper warmup period that is required to initialize all my indicators and to start trading at a specific date.
So let's calculate a ROC Indicator with the lenght 5 and start trading at the first of January.
warmup: int = 5
roc = pybroker.indicator('roc', lambda data: pandas_ta.roc(Series(data.close), length=warmup))
start_date: datetime = datetime(2023, 1, 1, tzinfo=pytz.timezone('America/New_York'))
end_date: datetime = datetime(2023, 2, 1, tzinfo=pytz.timezone('America/New_York'))
strategy: Strategy = Strategy(
Alpaca(os.getenv('ALPACA_KEY_ID'), os.getenv('ALPACA_SECRET')),
start_date, end_date,
StrategyConfig(initial_cash=10000, exit_on_last_bar=True)
)
strategy.set_before_exec(before_exec)
strategy.add_execution(exec_fn, ['IYY', 'IWM', 'IVV'], indicators=[roc])
result: TestResult = strategy.backtest(timeframe='1d', warmup=warmup)
So we need 5 periods to get the ROC(5) indicator calculated. This takes until the 2023-01-10 and the first trade is opened on the 2023-01-11:
date cash equity margin market_value pnl unrealized_pnl fees
2023-01-03 05:00:00 10000.00 10000.00 0.00 10000.00 0.00 0.0 0.0
2023-01-04 05:00:00 10000.00 10000.00 0.00 10000.00 0.00 0.0 0.0
2023-01-05 05:00:00 10000.00 10000.00 0.00 10000.00 0.00 0.0 0.0
2023-01-06 05:00:00 10000.00 10000.00 0.00 10000.00 0.00 0.0 0.0
2023-01-09 05:00:00 10000.00 10000.00 0.00 10000.00 0.00 0.0 0.0
2023-01-10 05:00:00 10000.00 10000.00 0.00 10000.00 0.00 0.0 0.0
2023-01-11 05:00:00 30.70 10082.50 0.00 10082.50 82.50 0.0 0.0
My original intention was to use 'Strategy.backtest(start_date, end_date, ...)' to start trading at the 2023-01-01 and to use Pyhtons 'timedelta(days=warmup) to shift the data fetching so the warmup period is substracted from the start_date.
In this way we could start trading exactly at the 2023-01-01 (where stock exchanges closed so it was in reality the 2023-01-03) .
warmup: int = 5
roc = pybroker.indicator('roc', lambda data: pandas_ta.roc(Series(data.close), length=warmup))
start_date: datetime = datetime(2023, 1, 1, tzinfo=pytz.timezone('America/New_York'))
end_date: datetime = datetime(2023, 2, 1, tzinfo=pytz.timezone('America/New_York'))
strategy: Strategy = Strategy(
Alpaca(os.getenv('ALPACA_KEY_ID'), os.getenv('ALPACA_SECRET')),
start_date - timedelta(days=warmup), end_date,
StrategyConfig(initial_cash=10000, exit_on_last_bar=True)
)
strategy.set_before_exec(before_exec)
strategy.add_execution(exec_fn, ['IYY', 'IWM', 'IVV'], indicators=[roc])
result: TestResult = strategy.backtest(start_date, end_date, timeframe='1d', warmup=warmup)
But doing so this exception is thown:
Traceback (most recent call last):
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 582, in _validate_comparison_value
self._check_compatible_with(other)
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 461, in _check_compatible_with
self._assert_tzawareness_compat(other)
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 694, in _assert_tzawareness_compat
raise TypeError(
TypeError: Cannot compare tz-naive and tz-aware datetime-like objects.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 1054, in _cmp_method
other = self._validate_comparison_value(other)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 585, in _validate_comparison_value
raise InvalidComparison(other) from err
pandas.core.arrays.datetimelike.InvalidComparison: 2023-01-01 00:00:00-04:56
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pirat/PycharmProjects/pybroker-experiments/main.py", line 120, in <module>
main()
File "/home/pirat/PycharmProjects/pybroker-experiments/main.py", line 110, in main
result: TestResult = strategy.backtest(start_date, end_date, timeframe='1d', warmup=warmup)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1060, in backtest
return self.walkforward(
^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1179, in walkforward
df = self._filter_dates(
^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1340, in _filter_dates
df = _between(df, start_date, end_date).reset_index(drop=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 79, in _between
(df[DataCol.DATE.value].dt.tz_localize(None) >= start_date)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/common.py", line 72, in new_method
return method(self, other)
^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arraylike.py", line 62, in __ge__
return self._cmp_method(other, operator.ge)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/series.py", line 6243, in _cmp_method
res_values = ops.comparison_op(lvalues, rvalues, op)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/array_ops.py", line 273, in comparison_op
res_values = op(lvalues, rvalues)
^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/common.py", line 72, in new_method
return method(self, other)
^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arraylike.py", line 62, in __ge__
return self._cmp_method(other, operator.ge)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 1056, in _cmp_method
return invalid_comparison(self, other, op)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/invalid.py", line 36, in invalid_comparison
raise TypeError(f"Invalid comparison between dtype={left.dtype} and {typ}")
TypeError: Invalid comparison between dtype=datetime64[ns] and datetime
So I came across an other idea to move the backtest start_date into future remove the warmup parameter, but this resulted into the same exception:
result: TestResult = strategy.backtest(start_date + timedelta(days=5), end_date, timeframe='1d')
So I came to the conclusion that Strategy start_date and end_date should not be different then Strategy.backtest(start_date, end_date) and therefore I dont understand why this redundancy is kept?
So idealy I would like to simply fetch i.e 1 month more data then required from the start_date from Alpaca and then start backtesting at exaclty the start_date (or the next candle). Is it possible to achive this?
Example is here: https://github.com/Pirat83/pybroker-experiments/blob/master/main.py
The start_date and end_date passed to the Strategy constructor specifies the date range to download from a DataSource. The start_date and end_date passed to Strategy#backtest can be a subset of that date range. The warmup parameter specifies the number of bars to skip after the start_date before running the Strategy.
Okay so it should work like desiered. I will give it a try. Thank you very much.