Multiple timeframe merge
surfablebot opened this issue · comments
Please reformat your post properly (wrap code snippets in tripple backticks (`)).
It's also completely unclear what you're trying to accomplish with this issue - it seems quite messy and some "i did this and then that" - but it's unclear what the actual issue is you're seeing / trying to fix.
"dropping" is not the same as what #4073 points at points at.
That's shifting the timeframes so the candle close dates align. Not doing that will delay the signal by 1 minor candle.
in backtesting - this is not problematic - in live, it'll be, as you'll be using the longer candle not at 00:00 - but only at 00:15 (assuming 15m/1h candles) - which is 15m later than when the candle would've been available.
Please find a detailed explanation of this problem here.
It's simply because the dates we work with are candle open dates.
It's not a problem - and nothing that should be optional. Systems not respecting this must be considered wrong - and should therefore be fixed.
considering it's binance - it's most likely a missconfiguration of the indicator - not a systematic problem - although i have to admit, i've never used indicators from a different timeframe, and am not sure how that'd be added.
I don't really understand why you've removed the issue content / your response now.
it's a valid question (this IS a difficult topic) - and a very frequent point of concern.
seeing that binance missaligns (maybe in certain conditions?) may also help other users - would the content have remained.
The issue for me was caused by removal/shifting of the candle in resampled_merge and merge_informative_pair. Not binance.
You are doing an amazing project to help all, I felt I should not get into an argument with you.
i don't really see it as an argument though - it's a fact that without that logic, the signals will be delayed by 1 "minor candle" (smaller timeframe). That's not subjective, but a mathematical fact as we're dealing with candle open dates - but can use the candle as soon as it closes.
If that delay is what you desire/need - then you can pretty easily use pd.merge directly - without relying on the helper function.
The helper functions are designed to help users merge correctly, without causing delays in their signals. if the delay is intentional - then the helper-functions may not be the correct approach for that strategy.
Having such a thing optional will be problematic - as it'll have users experiment with parameters they don't really understand the implications of.
My 2 cents will be to avoid touching data and let the user manipulate the raw data as much as possible. It is mathematically significant in multi frequency cascading (i.e. issuing a false entry/exit signal), but I agree it might cause users a headache :)
PS. I am super impressed with your responsiveness and diligence!!!
The fix you're applying is unclear to me (the original issue didn't really show what the precise change was) - but as it's backtesting only - i think you're risking lookahead bias (which will usually show high profits in backtesting - but will then disappoint in live).
now that's not to say this must be the case - but it's a possibility you shouldn't neglect
I found the problem in live run and live data. The change we put in to show the raw candle in Telegram triggered me to dig through the data with a fine comb. I will post you if the live results differ, so far it is working like a charm.
The fix is to add a flag i.e. raw=False
def resampled_merge(original: DataFrame, resampled: DataFrame, fill_na=True, raw=False, prefix=''):
to bypasses the following logic in resampled merge & informative merge:
if raw == False:
(current logic)
if original_int < resampled_int:
# Subtract "small" timeframe so merging is not delayed by 1 small candle.
# Detailed explanation in https://github.com/freqtrade/freqtrade/issues/4073
resampled["date_merge"] = (
resampled["date"] + to_timedelta(resampled_int, "m") - to_timedelta(original_int, "m")
)
.........
else:
resampled["date_merge"] = resampled["date"]
Plus a cosmetic change to add a variable called prefix=''. Helping with contextualized time frequency naming e.g. primary, secondary, etc
if prefix != ''
resampled.columns = [f"{prefix}_{col}" for col in resampled.columns]
else: (the current naming convention)
resampled.columns = [f"resample_{resampled_int}_{col}" for col in resampled.columns]