Add indices to speed up `deleteRealtimeData` store method
0xOlias opened this issue · comments
I've observed timeouts in the deleteRealtimeData
query when running against a large database (~20M rows in logs
). The queries in that database transaction use the following columns:
blocks.chainId
blocks.number (already have an index)
transactions.chainId
transactions.blockNumber
logs.chainId (already have an index)
logs.blockNumber
rpcRequestResults.chainId
rpcRequestResults.blockNumber
logFilterIntervals.startBlock
logFilterIntervals.endBlock
factoryLogFilterIntervals.startBlock
factoryLogFilterIntervals.endBlock
I have a hunch that as long as each table has an index on block number (only one for the interval tables), we'll be fine. The query always passes a block number very close to tip, so if the planner is able to use that index first, scans are probably fine for the remainder (eg chain ID).
Another (creative, heh) idea would be to take advantage of the existing blocks.number
index.
DELETE FROM "transactions" where "blockHash" in (SELECT "hash" from "blocks" WHERE "number" > $1);
I think this would get us the same planner optimization described above without adding an index on transactions.blockNumber
(which would only get used in this one infrequent query).
Before moving forward with a fix, it would be smart to isolate the slowest query within the transaction.