UI: Respond to state change signal in all tabs
dexX7 opened this issue · comments
- 18) Respond to state change signal in "Overview" tab
- 19) Respond to state change signal in "Balances" tab
- 20) Respond to state change signal in "Send" tab
- 21) Respond to state change signal in "Transactions" tab
- 22)
Respond to state change signal in "Toolbox" tabedit: it pulls, so no auto-update required.
As mentioned via email:
I tested every tab and instant feedback is given everywhere, if the own client is the active party, wether it's balances or something else and this seems to be very solid.
Testing with two clients was a bit more tricky, but the behavior is as follows:
Simple Send:
- A sends some MSC to B
- B receives the Bitcoin transaction, but, while unconfirmed, has no information about the MSC part, nor a balance update or alike
- After a block is mined, all balances update and the transaction shows up in the Omni history
Send to Owners:
- A sends to MSC owners
- B gets no feedback of any kind
- After a block is mined the overview tab updates, while the others need a manual refresh, e.g. by selecting a currency explicitly to force an update, which is not possible for the transaction history
This can create a very strange impression, when considering the situation without signals:
- Block 105: A sends MSC to B, B sees a Bitcoin transaction, but no MSC
- Block 106: The MSC shows up for B, A sends MSC to MSC owners
- Block 107: B sees no update in the tx history, even though a block was mined
- Block 108: B still sees no update in the tx history, until A sends some MSC to B - the STO transaction pops up, triggered by the incoming BTC, but the MSC send remains invisible at this point
- Block 110: Finally B sees the STO transaction as well as the Simple Send
Quick info, since you're online: I'm currently testing signals on all tabs, basically just connecting alongside the balanceChanged signal to refreshOmniState, to handle internal and external state updates.
Quick info, since you're online: I'm currently testing signals on all tabs, basically just connecting alongside the balanceChanged signal to refreshOmniState, to handle internal and external state updates.
Lovely, lovely, lovely - thank you mate :) Let me know how it goes, I'm just still sorting the column width and resizing issue on Omni transaction history...
It works, but I'm not sure about approach. There are two models: WalletModel and ClientModel. The OmniStateChanged signal is associated with the ClientModel, which is only available for the "Overview" tab, while the others are only linked to the WalletModel.
Is there any harm in connecting the other tabs with the ClientModel as well?
Or alternatively: move the signal into the WalletModel.
Yeah, originally I think the idea was for Bitcoin Core to support multiple wallets so you have the client model which handles things blockchain related (notify of new blocks etc) and the wallet model which handles things related to the wallet.
I felt clientmodel was the right place, as the signal was notifying of a new block with Omni transactions (wallet contents were irrelevant) but we can look at a move sure.
There is also something else to bear in mind - we still need to update certain parts of the display on a block change even without any Omni transactions (for example the icons indicating number of confirmations).
Thanks
Z
Yes, I ended up to subscribe to both models, to continue to handle outgoing, unconfrimed transactions as well. PR incoming in a few minutes.
I have the impression all those signals and extreme processing of everything has a significant effect on the performance, at least when running the test suite the UI appears to freeze for a moment. Not sure about the real world impact, but this is certainly something to address rather sooner than later, to avoid creating huge overheads.
Edit: oh, actually, I noticed this with a build without this PR. I'm
100% agree, efficiency is nowhere to be found at this stage sadly. The processing is heavy, and there are likely lots of avenues for optimization. On top of that I think we're updating probably more often than we need to.
I usually do things with timers to time each function, but holistically as an overall thing that's time consuming and cumbersome - this may sound like a bit of a dumb question, but are there any profilers out there that could give us a list of functions as they're called and how long they take? Alternatively I can manually time each function with timers and printfs etc.
Thanks
Z
FYI updateHistory for example takes 0.38 secs.
On top of that I think we're updating probably more often than we need to.
Yes, way to often, and even when not required. But this is no deal-breaker, because the UI can still be optimized, and it's fine, as long as it doesn't break, at least in my opinion. It would be different, if we'd slow down the server deamon though.
Profiling
This would really help to identify bottlenecks. Unfortunally I'm not into this topic, but I stumbed upon Valgrind a few times, and as low level tool there is gprof, as far as I know.
FYI updateHistory for example takes 0.38 secs.
As far as I can see each time UpdateHistory() is called transactions are fetched from the wallet and parsed. Let me say it this way: I have a few ideas to optimize this, or to optimize how updates are handled, or to work further towards plain transaction objects and serialization (mastercoin-MSC#283 (comment)), but I'm not sure, how to prioritize here.
As mentioned at the beginning, I think it would be best to solve all cricial issues first, e.g. wrong order, missing updates, if there are any, ..., then start to cleanup, and finally start to optimize or replace components. :)
What exactly did you measure?
Yep agreed, let's regroup on this for 0.0.9.2 (or if it's major surgery, perhaps the UI for 0.0.10 since we'll have extra stuff for MetaDEx there anyway).
I just stuck a boost timer at the beginning of the function, and then pumped it out at the end of the function (I like using timer because it gives elapsed time not CPU time which can be skewed eg with thread sleeps and so on).
Completely agree on the criticals, I've tried to run through all the low hanging fruit and think that's pretty much it now - all that's left is to address the criticals relating to ordering and signalling which I'm diving into now.
Just wanted to say thanks for all your help on this dude and how much I appreciate it - with all the feedback over the last week I've been able to really make some progress so just wanted to say thanks :)
Z
Thanks for the feedback. :)
There are basically three broader problems, all somewhat connected to each other:
- Pulling and parsing a part or the whole history each time adds a significant overhead of many, many results that were basically already available
- The number of STO related iterations is
nTxs * nFilteredReceipts * nReceipts, if I'm not mistaken, and with only a few larger ones huge delays could be created. - Passive effects in general are a troublesome. It's not just STO, but we also have exodus purchases, additional grants via crowdsale bonus, dex trading, etc.
The problem is by no means that the building blocks are bad, and each step is quite fast, but it seems like tackling the problem from the wrong direction. Instead, we may try to create OmniTxHistoryEntry objects which are stored/cached and inserted, updated or deleted, once it's state changes. For most parts those objects should be immutable, except the number of confirmations, ...
... so what would be the best way to describe effects? A Simple Send is static and straight forward - a movement of tokens from A to B. A creation of a property with fixed supply is somewhat straight forward as well, but how about crowdsales for example?
There is an upside though: this is pretty related to something that might become an audit ledger at some point.
Let's use this issue for further discussion.
I really have a bad feeling about the impact of the STO handling, though I did not yet test it on mainnet, but I counted nearly 30000 iterations of txhistorydialog.cpp#L297-L341 while running the regtests. My concern is that one may spam the network with large STO transactions and basically shutdown all Qt clients.
Wow. Yikes. Well that's obviously not acceptable!
Another note: we are pretty wasteful regarding transaction processing in general, and I'm referring to things like going through all Omni transactions to find one, or to count them, etc., so we likely get away with this.
It basically comes down to the following:
We need access to transactions in some form, which ultimately provides:
- bool valid (or maybe not.. I don't support invalid transactions.. :P)
- bool bInbound
- std::string txTimeStr
- std::string displayType
- std::string displayAddress
- std::string displayAmount
- std::string txidStr
- bool fundsMoved
Maybe with raw values instead of strings, but this is a minor detail. The number of confirmations can be updated or fetched on the fly, when a new block is found.
What if we do a simple check such as CheckIfTransactionAffectsMe() once a new transaction is processed, say for example in mastercore_handler_tx() after mp_obj.interpretPacket(), and if true, the transaction is transformed into an object with the values stated above and held in memory?
The UI could then use those objects, instead of parsing and processing everything over and over again.
During startup, while the cache is still empty, the most recent transactions could be fully re-processed.
Edit: ideally, this would somehow be isolated to be only done, if the UI is enabled, to avoid any impact on pure server builds.
Imho it works. And we may start a new thread when it comes to creating a new model, etc., but for now, the intiial issue is solved.