Giters

nautechsystems / nautilus_trader

A high-performance algorithmic trading platform and event-driven backtester

Home Page:https://nautilustrader.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Order Filled After Cancellation Request Triggers Backtest Failure

rsmb7z opened this issue · comments

commented

Bug Report

An order cancellation request is made, and an OrderPendingCancel event is received. However, before the cancellation completes, the order is filled, triggering an OrderFilled event with last_qty=0. Following this, the Backtest system disconnects and raises the following exception:

2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: Canceling 1 open orders for SPY.ARCA
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderPendingCancel(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1152, venue_order_id=ARCA-1-1147, account_id=ARCA-001, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: [CMD]--> CancelOrder(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1152, venue_order_id=ARCA-1-1147)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderFilled(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1152, venue_order_id=ARCA-1-1147, account_id=ARCA-001, trade_id=ARCA-1-1064, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, order_side=BUY, order_type=MARKET, last_qty=0, last_px=131.44 USD, commission=1.00 USD, liquidity_side=TAKER, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: Disconnecting...
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: Disconnected
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: STOPPED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataEngine: STOPPED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.RiskEngine: STOPPED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.ExecClient-ARCA: Disconnecting...
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.ExecClient-ARCA: Disconnected
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.ExecClient-ARCA: STOPPED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.ExecEngine: STOPPED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.OrderEmulator: STOPPED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: DISPOSED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataEngine: DISPOSED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.RiskEngine: DISPOSED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.ExecClient-ARCA: DISPOSED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.ExecEngine: DISPOSED
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.MessageBus: Closed message bus
2006-04-28T20:15:00.300000000Z [ERROR] BACKTESTER-001.BACKTESTER-001: InvalidStateTrigger('RUNNING -> DISPOSE') state RUNNING
2006-04-28T20:15:00.300000000Z [ERROR] BACKTESTER-001.BACKTESTER-001: InvalidStateTrigger('RUNNING -> DISPOSE_COMPLETED') state RUNNING
Traceback (most recent call last):
  File "nautilus_trader\\backtest\\engine.pyx", line 928, in nautilus_trader.backtest.engine.BacktestEngine.run
  File "nautilus_trader\\backtest\\engine.pyx", line 1120, in nautilus_trader.backtest.engine.BacktestEngine._run
  File "nautilus_trader\\backtest\\exchange.pyx", line 836, in nautilus_trader.backtest.exchange.SimulatedExchange.process
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 671, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine.process_order
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 775, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine.process_order
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 862, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine._process_market_order
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 1493, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine.fill_market_order
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 1717, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine.apply_fills
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 1780, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine.fill_order
  File "nautilus_trader\\backtest\\matching_engine.pyx", line 2292, in nautilus_trader.backtest.matching_engine.OrderMatchingEngine._generate_order_filled
  File "nautilus_trader\\common\\component.pyx", line 2287, in nautilus_trader.common.component.MessageBus.send
  File "nautilus_trader\\execution\\engine.pyx", line 600, in nautilus_trader.execution.engine.ExecutionEngine.process
  File "nautilus_trader\\execution\\engine.pyx", line 612, in nautilus_trader.execution.engine.ExecutionEngine.process
  File "nautilus_trader\\execution\\engine.pyx", line 897, in nautilus_trader.execution.engine.ExecutionEngine._handle_event
  File "nautilus_trader\\execution\\engine.pyx", line 1049, in nautilus_trader.execution.engine.ExecutionEngine._handle_order_fill
  File "nautilus_trader\\execution\\engine.pyx", line 1088, in nautilus_trader.execution.engine.ExecutionEngine._open_position
  File "nautilus_trader\\model\\events\\position.pyx", line 366, in nautilus_trader.model.events.position.PositionOpened.create_c
  File "nautilus_trader\\model\\events\\position.pyx", line 325, in nautilus_trader.model.events.position.PositionOpened.__init__
AssertionError

Expected Behavior

Should handle the sitation gracefully.

Actual Behavior

Raises design time assertion check here.

Specifications

  • OS platform: Windows 11
  • Python version: Python 3.11
  • nautilus_trader version: develop (latest)
commented

The issue arises from an order inaccurately flagged as PARTIALLY_FILLED despite being fully executed, triggering anomalous responses within the backtester. Key events include:

  • Order erroneously reported as PARTIALLY_FILLED after complete execution (76 units of SPY.ARCA at 131.43 USD).
  • Discrepancy highlighted by remaining_qty=0.
  • Cancellation attempt on fully filled order; followed by a fill event reporting last_qty=0 at 131.44 USD.
  • Triggers assertion error related to PositionOpened event due to PositionSide.FLAT.
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderInitialized(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, side=BUY, type=MARKET, quantity=76, time_in_force=GTC, post_only=False, reduce_only=True, quote_quantity=False, options={}, emulation_trigger=NO_TRIGGER, trigger_instrument_id=None, contingency_type=NO_CONTINGENCY, order_list_id=None, linked_order_ids=None, parent_order_id=None, exec_algorithm_id=None, exec_algorithm_params=None, exec_spawn_id=None, tags=None)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: [CMD]--> SubmitOrder(order=MarketOrder(BUY 76 SPY.ARCA MARKET GTC, status=INITIALIZED, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=None, position_id=None, tags=None), position_id=SPY.ARCA-SimpleCrossOverStrategy-000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderSubmitted(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, account_id=ARCA-001, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA margin_init=0.00 USD
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: Updated AccountState(account_id=ARCA-001, account_type=MARGIN, base_currency=USD, is_reported=False, balances=[AccountBalance(total=9_992_238.54 USD, locked=0.00 USD, free=9_992_238.54 USD)], margins=[], event_id=72c3e121-b80b-46de-9abd-c755ea5ed02e)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderFilled(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, account_id=ARCA-001, trade_id=ARCA-1-1066, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, order_side=BUY, order_type=MARKET, last_qty=76, last_px=131.43 USD, commission=1.00 USD, liquidity_side=TAKER, ts_event=1146255300300000000)
******************************************************************************************************************************
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: Order MarketOrder(BUY 76 SPY.ARCA MARKET GTC, status=PARTIALLY_FILLED, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, tags=None) is partially filled
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: order.quantity=Quantity('76'), order.filled_qty=Quantity('76'), order.leaves_qty=Quantity('0')
******************************************************************************************************************************
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA net_position=0
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA margin_maint=0.00 USD
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: Updated AccountState(account_id=ARCA-001, account_type=MARGIN, base_currency=USD, is_reported=False, balances=[AccountBalance(total=9_992_238.54 USD, locked=0.00 USD, free=9_992_238.54 USD)], margins=[], event_id=2535e8cf-deb9-4ec2-b906-2b33649c8dad)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] PositionClosed(instrument_id=SPY.ARCA, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, account_id=ARCA-001, opening_order_id=O-20060428-1900-001-000-1143, closing_order_id=O-20060428-2015-001-000-1145, entry=SELL, side=FLAT, signed_qty=-0.0, quantity=0, peak_qty=76, currency=USD, avg_px_open=131.12526315789472, avg_px_close=131.43, realized_return=-0.00232, realized_pnl=-26.01 USD, unrealized_pnl=0.00 USD, ts_opened=1146250800300000000, ts_last=1146255300300000000, ts_closed=1146255300300000000, duration_ns=4500000000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: Canceling 1 order for SPY.ARCA (Place New Order)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderPendingCancel(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, account_id=ARCA-001, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: [CMD]--> CancelOrder(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA margin_init=0.00 USD
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: Updated AccountState(account_id=ARCA-001, account_type=MARGIN, base_currency=USD, is_reported=False, balances=[AccountBalance(total=9_992_237.54 USD, locked=0.00 USD, free=9_992_237.54 USD)], margins=[], event_id=bda87586-27fd-4833-91e2-e4060a7fde7c)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderFilled(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, account_id=ARCA-001, trade_id=ARCA-1-1067, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, order_side=BUY, order_type=MARKET, last_qty=0, last_px=131.44 USD, commission=1.00 USD, liquidity_side=TAKER, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: Disconnecting...
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: Disconnected
commented

Hi @rsmb7z

Thanks for the report, I'm assuming you're on the latest develop given your recent PR's?

Are you able to tell me the precisions for the data and instrument?

The logic which determines the fill status is here in the order base:
https://github.com/nautechsystems/nautilus_trader/blob/develop/nautilus_trader/model/orders/base.pyx#L1049

    cdef void _filled(self, OrderFilled fill):
        if self.filled_qty._mem.raw + fill.last_qty._mem.raw < self.quantity._mem.raw:
            self._fsm.trigger(OrderStatus.PARTIALLY_FILLED)
        else:
            self._fsm.trigger(OrderStatus.FILLED)

So I'm guessing there is some small fractional quantity remaining, and the only thing I can think would cause that would be an instrument with size precision greater than zero, or possibly something in the data affecting the fills.

commented

Are you able to tell me the precisions for the data and instrument?

It is SPY.ARCA with size_precision=0.

So I'm guessing there is some small fractional quantity remaining, and the only thing I can think would cause that would be an instrument with size precision greater than zero, or possibly something in the data affecting the fills.

Yes you are right, its due to size precision because Trade bar is divided by 4 to make TradeTick size and causing fractional error. I am fixing that in TradeTickWrangler to round according to precision.

For this report, I think check for precision_qty in addition to raw for Partially filled cases maybe more helpful for debuging/logging.

commented

We also have this validation in the matching engine, which I thought might prevent this sort of thing:
https://github.com/nautechsystems/nautilus_trader/blob/develop/nautilus_trader/backtest/matching_engine.pyx#L1633

            # Validate size precision
            if fill_qty.precision != self.instrument.size_precision:
                raise RuntimeError(
                    f"Invalid size precision for fill {fill_qty.precision} "
                    f"when instrument size precision is {self.instrument.size_precision}. "
                    f"Check that the data size precision matches the {self.instrument.id} instrument"
                )
commented

if fill_qty.precision != self.instrument.size_precision:

The reason this might not be working in this case is because TradeTick has correct size_precision assigned when created using TradeTick.from_raw_c and raw value was not rounded according to precision. I understand adding any check on object creation may lead to performance issues though.

This was happening where size_precision is correct but raw value is not.

from nautilus_trader.core.rust.model import AggressorSide
from nautilus_trader.model.identifiers import TradeId
from nautilus_trader.test_kit.providers import TestInstrumentProvider
instrument = TestInstrumentProvider.equity("SPY", "ARCA")
tick = TradeTick.from_raw(
	instrument.id,
	468010000000,
	instrument.price_precision,
	4683750000000,
	instrument.size_precision,
	AggressorSide.NO_AGGRESSOR,
	TradeId('1'),
	1704488400000000000,
	1704488400000000000,
)

>>> tick.size
Quantity('4684')
>>> tick.size.raw
4683750000000  <--- should have been 4684000000000
commented

Closing this in anticipation of the fact that its a bug in TradeTickDataWrangler.process_bar_data where the data is not being produced to spec where the raw value should match the precision.