consensus indicator used in strategy
tomismrkolj opened this issue · comments
Hi,
my brother and I have been working on consensus based strategy. Buy signals if hyperopted are nice. The problem we have encountered is heavy bot lag on rpi4, atom NAS and pc as well. During live trading about 2 - 5 minute delay was observed between bot logs and binance order placing. Sometimes orders weren't placed at all, ROI was not triggered and stoplimit orders were not cancled and reopened after 60s on binance to allow for trailing.
At first we had process_only_new_candles=false, but tested with =True and at least on atom NAS bot was placing acting quite ok, except for first 2 minutes of each candle, when indicators and scores were calculated.
On rpi4 each freqtrade instance in docker uses 100% of CPU time regardless of process_only_new_candles setting.
here is strategy file:
pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
--- Do not remove these libs ---
import numpy as np # noqa
import pandas as pd # noqa
from pandas import DataFrame
from freqtrade.strategy.interface import IStrategy
--------------------------------
Add your lib to import here
import talib.abstract as ta
import freqtrade.vendor.qtpylib.indicators as qtpylib
from technical.consensus import Consensus
class strategijaconsensus(IStrategy):
"""
This is a strategy template to get you started.
More information in https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md
You can:
:return: a Dataframe with all mandatory indicators for the strategies
- Rename the class name (Do not forget to update class_name)
- Add any methods you want to build your strategy
- Add any lib you need to build your strategy
You must keep:
- the lib in the section "Do not remove these libs"
- the prototype for the methods: minimal_roi, stoploss, populate_indicators, populate_buy_trend,
populate_sell_trend, hyperopt_space, buy_strategy_generator
"""
# Strategy interface version - allow new iterations of the strategy interface.
# Check the documentation or the Sample strategy to get the latest version.
INTERFACE_VERSION = 2
# Minimal ROI designed for the strategy.
# This attribute will be overridden if the config file contains "minimal_roi".
minimal_roi = {
"0": 0.10607,
"88": 0.06485,
"242": 0.0211,
"777": 0
}
# Optimal stoploss designed for the strategy.
# This attribute will be overridden if the config file contains "stoploss".
stoploss = -0.0203
# Trailing stoploss
trailing_stop = True
trailing_only_offset_is_reached = False
trailing_stop_positive = 0.01004
trailing_stop_positive_offset = 0.01115 # Disabled / not configured
# Optimal timeframe for the strategy.
timeframe = '30m'
# Run "populate_indicators()" only for new candle.
process_only_new_candles = True
# These values can be overridden in the "ask_strategy" section in the config.
use_sell_signal = True
sell_profit_only = False
ignore_roi_if_buy_signal = False
# Number of candles the strategy requires before producing valid signals
startup_candle_count: int = 30
# Optional order type mapping.
order_types = {
'buy': 'limit',
'sell': 'limit',
'emergencysell': 'market',
'forcesell': 'market',
'stoploss': 'market',
'stoploss_on_exchange': True,
'stoploss_on_exchange_interval': 60,
'stoploss_on_exchange_limit_ratio': 1.0
}
# Optional order time in force.
order_time_in_force = {
'buy': 'gtc',
'sell': 'gtc'
}
plot_config = {
# Main plot indicators (Moving averages, ...)
'main_plot': {
'tema': {},
'sar': {'color': 'white'},
},
'subplots': {
# Subplots - each dict defines one additional plot
"MACD": {
'macd': {'color': 'blue'},
'macdsignal': {'color': 'orange'},
},
"RSI": {
'rsi': {'color': 'red'},
}
}
}
def informative_pairs(self):
"""
Define additional, informative pair/interval combinations to be cached from the exchange.
These pair/interval combinations are non-tradeable, unless they are part
of the whitelist as well.
For more information, please consult the documentation
:return: List of tuples in the format (pair, interval)
Sample: return [("ETH/USDT", "5m"),
("BTC/USDT", "15m"),
]
"""
return []
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Adds several different TA indicators to the given DataFrame
Performance Note: For the best performance be frugal on the number of indicators
you are using. Let uncomment only the indicator you are using in your strategies
or your hyperopt configuration, otherwise you will waste your memory and CPU usage.
:param dataframe: Dataframe with data from the exchange
:param metadata: Additional information, like the currently traded pair
:return: a Dataframe with all mandatory indicators for the strategies
"""
# Consensus strategy
# add c.evaluate_indicator bellow to include it in the consensus score (look at
# consensus.py in freqtrade technical)
# add custom indicator with c.evaluate_consensus(prefix=<indicator name>)
c = Consensus(dataframe)
c.evaluate_rsi()
c.evaluate_stoch()
c.evaluate_macd_cross_over()
c.evaluate_macd()
c.evaluate_hull()
c.evaluate_vwma()
c.evaluate_tema(period=12)
c.evaluate_ema(period=24)
c.evaluate_sma(period=12)
c.evaluate_laguerre()
c.evaluate_osc()
c.evaluate_cmf()
c.evaluate_cci()
c.evaluate_cmo()
c.evaluate_ichimoku()
c.evaluate_ultimate_oscilator()
c.evaluate_williams()
c.evaluate_momentum()
c.evaluate_adx()
dataframe['consensus_buy'] = c.score()['buy']
dataframe['consensus_sell'] = c.score()['sell']
return dataframe
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators, populates the buy signal for the given dataframe
:param dataframe: DataFrame populated with indicators
:param metadata: Additional information, like the currently traded pair
:return: DataFrame with buy column
"""
dataframe.loc[
(
(dataframe['consensus_buy'] > 34) &
(dataframe['volume'] > 0) # Make sure Volume is not 0
),
'buy'] = 1
return dataframe
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators, populates the sell signal for the given dataframe
:param dataframe: DataFrame populated with indicators
:param metadata: Additional information, like the currently traded pair
:return: DataFrame with buy column
"""
dataframe.loc[
(
(dataframe['consensus_sell'] > 88) &
(dataframe['volume'] > 0) # Make sure Volume is not 0
),
'sell'] = 1
return dataframe
I have to correct myself: also on pi4 cpu is loaded 100% at the beginning than load drops to even 7%. This is with docker. Is there a possibility that native rpi freqtrade build would load cpu even less?
in the end, every call you make to an "evaluate_*" method will calculate some indicator.
While some are quite optimized and run fast, others are not and require a loop (for example laguerre, but i guess it's not the only one).
For most indicators that use a loop in technical, it's required and not possible to calculate differently - so there's also very little room for optimization.
Now this happens for every pair ... so depending on your pairlist, this can take a while, and as its a loop, it'll be cpu intensive for sure.
You can try to remove one or the other of the evaluate* methods to check which one is the ressource consumer - but i suspect there'll not be much we can do to change or optimize that.
ok, my brother and me have done extensive tests. It seems there is cpu problem. Rpi 4 and NAS with Intel atom struggle with calculations at the beginning of each candle (and sometimes in between) and are few minutes late with buys and sells during this periods in comparison to pc with Intel i5. During live trading this means bot does not control the orders as it should.
Is there any chance that freqtrade would use multiple processor cores?
as said above, it'll be one of the indicators, not all.
No, freqtrade will not use multiple cores for indicator calculation.
@xmatthias what if we divided the pairlist into 3 or 4 bots, and ran them simultaneously? Would they interfere on the exchange?
it is a conceptual question:
does 1 bot with static parilist of 40 coin pairs perform the same as 4 bots with the same static list divided between them by 10 coin pairs. As i understand freqtrade in fact preforms operation on each coin pair sequentialy, so at each moment only one coin pair is under analysis.
- Are coin pairs independent of each other? If yes, then bots can be combined to deliver the same financial win/loss.
- should they share common wallet on exhcange or does each bot have to have its own subaccount wallet?
- stake amount is divided by 4. What about max open trades? Is it possible that some coinpairs that are together on one bot might use all trade "sockets", while another bot does not have any trades open. Therfore 1st bot is limited eventhough it might have more coin pairs with buy signals. So if we divide coinlist to 4 bots, is it necessarly true, that we should also divide max open trades by 4?
- if one large bot is divided to 4 smaller ones, do we have to perform hyperopt on each of them or do we use same strategy for all 4 bots?
crypto coins in general are not independend, but most depend on BTC performance.
As long as you're using static stake-amounts, running multiple bots on the same account should not be an issue (even if they'd trade the same pair).
it'll obviously be completely independent instances, so if you use the same (or different) strategy is up to you.