Giters

IntersectMBO / plutus-apps

The Plutus application platform

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Implement functions to build a balanced transaction with fees

Jeyhey opened this issue · comments

commented

Describe the feature you'd like

The only current way to sign a transaction that contains script UTxOs is to pass a completely balanced transaction including fees to the user's wallet (such as Nami). There is no current wallet that implements balancing transactions and according to an issue opened at the Nami GitHub repository such a feature cannot be expected in the close future (see here)

Plutus.Contract.Request does not provide a function to balance transactions for foreign wallets. Balancing, calculating fees, and submitting the transactions are done in one pass for mock wallets. There doesn't seem to be a way to specify the address of the wallet for which the balancing is desired. Also, there does not seem to be a way to calculate the fees and return the transaction without also submitting it.

A possible solution needs to either provide a way to instantiate a Contract with a given wallet address or to provide a function such as balanceTxFor that takes a wallet address as argument. In both cases an additional function calculateTxFees is needed for constructing the final transaction that can be handed over to the user's wallet for signing.

Describe alternatives you've considered

There is no current alternative to passing the final transaction to the user's wallet to my knowledge. If there will be one soon, I'd be happy to read more on this.

commented

Hi! Commented on the referenced issue. Before doing anything, I want to make sure that it's really not possible with Nami.

commented

Thank you for looking into this. The dApp connector of Yoroi will soon be released. I'm wondering if it will have a balancing function.

Independently of any wallet API, I'm wondering if my requested feature should not be implemented anyhow. It seems like a very basic thing. Even if not the currently recommended way of interacting with smart contracts, it could be helpful for debugging.

commented

For Yoroi, I'd guess not if they strictly follow CIP-0030. I created an issue in the CIPs github repo to get feedback on including a balanceTx function in light wallets: cardano-foundation/CIPs#192. This would be an extension to CIP-0030 which wallets can optionally abide to.

Back to your proposal. If I'm understanding correctly, you're proposing the PAB handles balancing a transaction using inputs from a wallet (independently of the one that was used to activate the contract). I guess we can investigate that. Ultimately, I think we should not implement a new balancing function and we'll probably end up reusing the one from cardano-wallet.

commented

Exactly. Thanks, this is great to hear. I am quite stuck at the moment and I think this feature would really help making the PAB more useful.

commented

Is there any workaround until the feature is available in PAB? I'm also stuck with balancing Txs with Nami wallet that include plutus data in witness set.

commented

Hi, I just wanted to check if there is any update on this. Not trying to pressure or anything. Just curious.

commented

Update:

This is temporary on hold.

So, balancing transactions through the PAB will require calling the balanceTransaction endpoint from cardano-wallet. However, we can't use balanceTransaction unless we already have a synced wallet in cardano-wallet. Since the goal is to balance transaction with utxos from external wallets (like Nami) and that browser wallets (like Nami) can't balance transactions which contain script inputs, there are two solutions:

  1. Fork cardano-wallet and modify the balanceTransaction so that we can pass an input of external utxos.
  2. Wait for the cardano-wallet team to release a cardano-wallet-core which contains the functionnality to balance transactions using external utxos.

Very have been discussions for the better option 2, but it might take a while. In the short term, option 1 seems better.

So, we'll work on it once we can.

commented

Thank you for the update. This is unfortunate. What do you think of forking the PAB as a third option ?

commented

I'd suggest considering ogmios' new EvaluateTx feature as a 3rd option to that end.

The command works similarly to the submission protocol except that it only runs the Phase-2 validations with unlimited budgets, and returns the actual execution budgets. Behind the scene, it gathers necessary information such as UTxO or protocol parameters from the node itself (although it allows passing an extra UTxO for input resolution).

This is particularly convenient in contexts such as light wallets that have ways to already preconstruct transactions but only needs to run the Plutus execution part.

commented

So, ogmious —a project with one contributor in the past month— implements a feature missing in the PAB which has 15 contributors in the past month ? Shouldn't the PAB just copy the approach of ogmious or use it as dependency ?

I lack the necessary insights to understand why it is so difficult to implement balancing transactions in the PAB. Even more, I don't think there is any information out there helping to investigate this issue. I thought that the most basic use case of the PAB was its use together with browser wallets. Seeing this being put on hold really leaves me puzzled.

I have a 4th option: I will implement my own balancing helper function that just uses a (fine tuned) rough estimate for the fees, hoping that most transactions will pass. I'll just accept that the users pay slightly higher transaction fees than necessary.

commented

You're genuinely asking why is it more difficult to pivot a project which has undergone years of development from 15 different people and multiple companies, than to implement a feature in a more recent, much more lightweight project maintained by a single individual?

I don't think your comment is really helping the discussion, and I don't want to turn this issue into a debate about software development strategy.

You've been asking for solutions and various solutions have been proposed. If you're now seeking for justifications and rationale behind product prioritization and development strategies, Github isn't the right place for that.

commented

So implementing this feature is a pivot ? I thought the integration of browser wallets was planned all along.

It is almost impossible to get into contact with the Plutus developer team. As a Cardano developer you are just left there waiting and wondering. And then when you want to get into a discussion there is only a limited exchange of ideas. I am not asking for much and certainly not expecting others to just solve my problems. I am invested in the Cardano ecosystem and would happily invest further time helping to make progress on this. As I said above, I already have some ideas how I could solve the issue for me.

You cannot blame me for not being helpful while giving me no chance to be so in the first place. I don’t think that there are many other developers out there investing as much time as me, in their spare time, into Plutus smart contracts. It is really disappointing to be treated this way.

commented

@Jeyhey I understand your frustrations. Things are moving slowly on this issue simply because we have other higher priority work that needs to be done.

I thought the integration of browser wallets was planned all along.

Unfortunately, no. Integration with Daedalus (desktop wallet) was planned all along which can very well balance transaction. The problem is browser wallets and they have been in scope only for the past few months.

What do you think of forking the PAB as a third option ?

I think that makes sense. If you have a viable solution to that problem on your fork, you can always do a PR upstream, and we can review it.

MELD already a temporary fork of cardano-wallet which contains option 1 described above: https://github.com/MELD-labs/cardano-wallet/commits/master

You could use this fork to run a WBE server, and use it only to call balanceTransaction and pass in the utxos from your browser wallet. Might be overkill to run WBE just for that, but it's a quick workaround for the time being.

commented

Thank you for the hint. MELD looks promising.

I have thought again about the right smart contract architecture and came to the conclusion that the main use case of the PAB should be to provide unbalanced transactions and that it should be the task of the wallet to balance it. I have written a blogpost with more details on this here: we-need-better-wallets-on-cardano

I still think that a method for providing balanced transactions would be useful for debugging purposes but to protect the users' privacy it would be best to move away from the current model of CIP-30 where the user grants total read access of the wallet to the dApp.

With Ogmios, there seems to be a service now that can be used by browser wallets for calculating the fees of transactions containing smart contracts. Do you know what would be needed for browser wallets to directly calculate the fees by evaluating the smart contracts ? Do you know of any development going into this direction ?

commented

@Jeyhey, browser wallets will always need to rely on some kind of service to evaluate transaction units since this depends on "moving targets" such as protocol parameters or UTXO entries. Thus, this is not something that can fully be done client-side without any access to the blockchain.

Having said that, there's a workstream which aims at improving GHCJS, which is a version of the Haskell compiler that targets Javascript as output. The end goal being to cross-compile the Plutus core libraries to Javascript in order to make the core building blocks available client-side. I can recommend following the work of angermann in that regard!

Regarding CIP-30 and privacy, note that we originally discussed the idea of roles and permissions to be granted to DApp. All-in-all however, the wallet remains in control of the information it sends out so, it can very much chose what to expose through the interface already (there's nothing forcing the wallet to give the entire UTXO set when one requests it).

There's also room for addendum and annexes, in particular regarding coin-selection. If you're up to it, I am glad to help you working on a new CIP to specify a coin-selection interface.

commented

@Jeyhey Your blog is a good read, thanks! I have two comments:

  • When an external service like PAB balances the transactions, the dApp interface can send just enough UTxOs to do it. And as @KtorZ said, a wallet doesn't need to expose the whole UTxO set to the dApp interface requesting "just enough" UTxOs for a transaction. If privacy is so much desired the wallet can ask for user confirmation before exposing information to build each transaction. As a user, I would just split wallets instead though.
  • PAB cannot scale on mainnet for arbitrary dApp logic, only a few patterns work at the moment. A more appropriate conclusion might be for everyone to join hands to build better tooling on all fronts, not just the wallets.

On another note, we've been scaling PAB on the mainnet for a month now and needed non-trivial efforts to do it. From your perspective, what is the current number one missing piece, and what should we do first to make instant contributions to PAB users?

I guess it could be anything from a markdown explaining how to run and scale PAB for dApp interaction on the mainnet, to adding Nami support to plutus-starter. We are actually building a dApp-starter with PAB, Nami, and hopefully, Hydra integration included in the future as well.

commented

Hi @kk-hainq, I'm happy you enjoyed reading my blogpost. Sorry for replying late.

1. CIP-30

  • Of course a wallet can do anything it wants, a standard cannot enforce compliance. But there is no point in establishing a standard and then encouraging non-compliance. A standard that cannot be applied needs revision.

  • To avoid any misunderstanding, if the wallet does not return all UTxOs it is not CIP-30 compliant as the standard defines for method api.getUtxos the following:

If amount is undefined, this shall return a list of all UTXOs (unspent transaction outputs) controlled by the wallet.

2. PAB scaling

  • I am not sure what you mean by "scaling PAB on the mainnet". It sounds interesting and I would really like to hear more about it.

3. Missing pieces in the short term

  • Regarding the missing pieces, I'm happy to give my view on this. I think before anything else we/you should focus on having a solution for deploying the current standard use case, i.e. browser wallet integration via a webpage. I recently watched the Plutus Pioneer Program videos and got surprised that it is already at its third iteration but it is still not showing how to deploy DApps. We only need one feature in cardano-wallet to have a working (though cumbersome) deployment architecture which consist of i) the PAB, ii) the chain index, iii) cardano-wallet, and iv) cardano-node. So, the first priority is to get this feature (balancing for foreign wallets) in cardano-wallet. Using the MELD-fork feels strange.

  • Second, with the architecture ready, I would expect a cli tool for creating smart contract projects (directory structure, needed files...). I don't know how complex it is to develop such tools. So I would be ok with a starter project that can be used as a template in the meantime. It is great to hear that you are working on this.

  • Third, while I understand why we need all these elements, I can't help but think that four separate pieces are overkill. I think the PAB, the chain index and cardano-wallet should be bundled into one component, in the sense that the DApp developer views them as one thing. I am not asking for repackaging or anything the like. I just want the documentation to be able to say: If you want to deploy smart contracts you need two components: a Cardano node and the PAB.

commented

@Jeyhey

  1. Sorry for the misunderstanding, I was talking about how one can hide the full UTxO set of the users from external balancing endpoints like from PAB, Cardano Wallet, or future Blockfrost, you name it. I also mentioned how we could extend CIP-30 to ask for user permissions (confirmation) for getUtxos and other sensitive functions (so the wallet cannot fake a large transaction to get most of the user UTxOs anyway) like how CIP-30 is asking for signTx and signData. I also concluded that if we asked for user confirmation too often it would yield bad UX and users might as well split wallets and only store enough in "hot" wallets if they consciously want privacy. Or both, where CIP-30 allows users to configure it in the wallet UI!

  2. Hopefully, we will be able to write a lot more about it. In general, plutus-chain-index is very slow and a utxosAt call in your PAB endpoint could take hours or forever to complete. We had to write our own indexing solutions on top of plutus-chain-index-core to scale it, and are actually encouraging people to do so as well. Extra code to maintain sucks, but it would be worth it for fast queries for both UI rendering and transaction building. We're already writing extra code to teach PAB how to build transactions, might as well write a bit more to teach it how to index and serve dApp data. Our current solution builds an external executable to replace plutus-chain-index though, a general framework right within PAB should take time to develop.

  3. All of your feedback makes a lot of sense and is indeed the direction we've been working towards. The ultimate goal is to design a super lean PAB that asks for just enough dApp-specific schema and code from developers to package and deploy a scalable infrastructure for the whole dApp. We have also been writing a starter dApp template with everything we have in production at the moment, and gradually improving it towards the ultimate goal above. I'll try my best to port most if not all of our internal ideas to this repository's issues.

commented

Has there been any update in regards to whether this is being worked on? From my perspective, this is the last thing that is needed by dApp developers to have a complete toolkit to develop production apps with the PAB. Right now, all the production dApps either have in-house versions of the PAB with custom transaction balancing functions, or they balance the transactions on the front-end via Cardano-serialization-lib (forked versions) which is super fickle, adds an unreasonable amount of overhead for developers to maintain and is just counter-intuitive to what a reasonable dApp backend should do.

As a dApp developer what I would like to see is:

  1. Front-end sends Datum, Redeemer, and a list of UTXOs all the information necessary to build the balancedTx to some PAB endpoint.
  2. PAB builds and balances the Tx.
  3. Front-end fetches the balanced tx from PAB, signs it, and submits it.

Clean and concise.

commented

@colll78 This is in progress. However, it might take a few sprints before it is completed.

commented

@kk-hainq Thank you for the details. I appreciate the discussion even though I cannot keep up the pace and reply as fast as you : )

What kind of indexing do you suggest for chain index ? Would additional indexes also increase the syncing of chain index with Cardano Node ? I'm running it since weeks for the testnet and I'm on the edge of giving up. It progresses extremely slowly. For big blocks it is barely faster (syncing around 1 to 3 blocks per 30 seconds) than the block generation itself. And then it pauses syncing regularly after around 0.1% progress for an hour or two. I wonder if syncing the mainnet is realistic at the current state. What would be really helpful would be to have the chain index DB available for direct download. Do you know if there is such a possibility?

Best regards,
J

commented

@Jeyhey In the last week, there were commits to make the chain-index syncing faster. It should take probably 1 day to sync on testnet right now.

commented

@koslambrou thx for the hint. I'll check this out. Sounds great

commented

@colll78 This is in progress. However, it might take a few sprints before it is completed.

@koslambrou The Ardana Apps team is building their stablecoin protocol's off-chain code using the PAB, and is therefore very interested in the status and progress of this issue. We'd be happy to help out on any development related to this.
Let me know if you would like to have a chat about anything we can help with!

Do you have an update on when the solution to this issue would be ready?

P.S.: We are also working on YubiHSM integration with the PAB, to allow projects to use an admin key which is stored securely on a production server. From what I gather, there are other projects interested in this. I will reach out with more information on that later :)

commented

@koslambrou, I tested the April 6th update of Plutus Chain Index. I am still struggling with the speed. Which setup do you use to get to full synchronization in about one day ?

I opened an issue here to separate it from this topic: #429

Update of April 29th: Now, after one week of syncing, progress stands at 86.25% with a change of 0.3% in the last 6 hours.