comparing demeter backtesting result to real positions results show some gaps

[lior david]

Hi,
I am trying to learn how to use Demeter and test some simple static strategies comparing to real-world positions and understand if my strategy works well and how close it is to real-world results.
my findings show gaps of 5-10% in collected pees calculation and also in position liquidity at the end of the test.

I am new to Python and Demeter so maybe I am doing something wrong.
is that an expected result? I saw in the code that you don't support fee calc when a tick is crossed, which may have an effect on accuracy, do you plan to support it in the future?

example:
i took this position 
usdc/eth 0.3%
24.10 7:39 - 29.10 18:00
https://etherscan.io/tx/0xe694c090428c8d104a5a9b7a6a10c9ae934de9a0df06a4de2d20719181d7a346
https://app.uniswap.org/pools/589914

demeter results:

usdc fee 150.73527 (vs 142)
eth fee 0.0792835 (vs 0.07613)

usdc in position 22488.34 (vs 25990)

eth in position 10.38382 (vs 11.02)

see the attached image for real results at the time of comparison.

I used demeter-fetch to get data from big query

[from]
chain = "ethereum" # polygon or ethereum
datasource = "big_query" # big_query or rpc or file
dapp_type = "uniswap" # uniswap or aave
[from.uniswap]
pool_address = "0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8"
[from.big_query] # keep this section according to from.datasource
start = "2023-10-24"
end = "2023-10-29"

[to]
type = "minute" # minute or tick or raw
save_path = "./usdc-eth-data"
multi_process = false # process in multi_process, defaut: False

thats the strategy and test file:

---

from datetime import date

import pandas as pd

from demeter import UniV3Pool, Actuator, MarketInfo, UniLpMarket, TokenInfo, Strategy, EvaluatorEnum
from demeter.download import ChainType
from strategy_ploter import plot_position_return_decomposition
from _decimal import Decimal
from typing import Dict

pd.options.display.max_columns = None
pd.set_option('display.width', 5000)

class MyStrategy3(Strategy):
def init(self):
super().init()

def initialize(self):
    market: UniLpMarket = self.markets[market_key]
    market.add_liquidity(1636.04, 1935.31)
    super().__init__()

if name == "main":
usdc = TokenInfo(name="usdc", decimal=6) # declare token0
eth = TokenInfo(name="eth", decimal=18) # declare token1
pool = UniV3Pool(usdc, eth, 0.3, usdc) # declare pool
market_key = MarketInfo("uni_market3")

actuator = Actuator()  # declare actuator
broker = actuator.broker
market = UniLpMarket(market_key, pool)

broker.add_market(market)
# add balances according to this tx
broker.set_balance(eth, 11.977767765444931017)
broker.set_balance(usdc, 24275.246863)

actuator.strategy = MyStrategy3()

market.data_path = "../usdc-eth-03-data"
market.load_data(ChainType.Ethereum.name.lower(),
                 "0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8", # ETH/USDC 0.3% uniswap v3
                 date(2023, 10, 25),
                 date(2023, 10, 29))
actuator.set_price(market.get_price_from_data())
actuator.run(
    evaluator=[EvaluatorEnum.MAX_DRAW_DOWN, EvaluatorEnum.ANNUALIZED_RETURNS]
)  # run test

# get result
evaluating_result: Dict[EvaluatorEnum, Decimal] = actuator.evaluating_indicator  # {<EvaluatorEnum.MAX_DRAW_DOWN: 3>: Decimal('0.04801148755391014037566625016'), <EvaluatorEnum.ANNUALIZED_RETURNS: 1>: Decimal('-0.9916890919693757317759809010')}
actuator.save_result("./res-hex",  # save path
                     account=True,  # save account status list as a csv file
                     actions=True)  # save actions as a json file and a pickle file

plot_position_return_decomposition(actuator.get_account_status_dataframe(),
                                   actuator.token_prices[eth.name],
                                   market_key)

---

the final status was:

net_value eth usdc uni_market3_net_value uni_market3_base_uncollected uni_market3_quote_uncollected uni_market3_base_in_position uni_market3_quote_in_position uni_market3_position_count

2023-10-29 23:59:00 46038.45896264917812562138567 0 4629.96960295986140984796065 41408.48935968931671577342502 150.7352701088900142474106575 0.07928350165347513303123274220 22488.34657493972420853693269 10.38382421897665466066069915 1

backtest-20231029-203206.action.json
backtest-20231029-203206.account.csv

[32ethers]

Oops. you add liquidity in initialize function, which you are adding liquidity before back test start, so you are holding the position at date(2023, 10, 25), but infact, your position was added in Oct-25-2023 07:38:59 PM +UTC, so there are 19 hours gap.

If you want to add mobility at a specific time, please use trigger.

class DemoStrategy(Strategy):

    def initialize(self):
        # Register a trigger
        self.triggers.append(AtTimeTrigger(time=datetime(2023,10,25,19,39), trigger_immediately=True, do=self.add_liquidity))

    def add_liquidity(self, row_data: RowData):
        market: UniLpMarket = self.markets[market_key]
        market.add_liquidity(1636.04, 1935.31)

Please modify your strategy and verify the output again. If there are still some gaps, please let me know.

[32ethers]

Close due to no further comment