EIP-2535: Diamonds
mudgen opened this issue · comments
EIP-2535 Diamonds exists here: https://eips.ethereum.org/EIPS/eip-2535
Below is a feedback and discussion of the standard.
This is great.
Would be curious to see a uniswap build using the Diamond pattern.
@androolloyd yes, me too!
I think you need to add the proper table to the top of this file. Wait, this is an issue not a PR. I'm confused are you trying to make a new EIP?
(Putting discussion here since I don't see an external link)
This looks interesting, albeit very ahead of it's time. You've clearly put a lot of thought into it, and it is very well written. I'm curious to see how this intersects with governance. My jerk reaction is that this is only useful in a specific application that includes permission-based "cut" control. But even that would introduce lots of systemic risk. For now I think we are all stuck with upgrades being controlled by a single source of control, like Maker and rDAI, which publish documentation to record the history of changes. I look forward to the day where this fine-grained control is reality, and small changes can be made on-the-fly without introducing additional risk!
Am I missing another use-case for this?
Edit: (follow-up) how can we isolate storage on the same permissioned system that we use to allow cuts?
@pi0neerpat thanks for your kind comments! The discussion is here. An official EIP for this standard is coming soon. It will be EIP 2535.
Companies are already using this kind of architecture (ERC1538) for their contracts, such as Caesar's Triumph, Enjin and others. It is real and in use today.
Enjin's NFT standard EIP 1155 recommends using an architecture like ERC1538 for upgrades.
I don't know why you think a diamond can't be controlled by a single source. The reference implementation of the standard is implemented that way -- only the single owner of the diamond can make changes. Maybe I am understanding you wrong? Authentication is not part of the standard, it can be fine grained or not.
A big use case is contracts with designs that exceed the max size of contracts, since diamonds don't have a max size. It is also very nice that while diamonds can be large their functionality can still be compartmented by facets. Particularly NFT contracts tend to exceed the max size limit, such as implementations of ERC721 and ERC998 and others that implement an NFT but also need to implement custom functionality for the application.
Permissions and authentication for cuts and access to storage variables can be handled in the same way as any other contract. Yes, if you wanted to, you could add various different permissions/authentications for different changes and handling different storage variables. Or you could keep it simple and not do that.
The owners(s) of an upgradeable contract have the ability to alter, add or remove data from the contract's data storage. Owner(s) of a contract can also execute any arbitrary code in the contract on behalf of any address. Owners(s) can do these things by adding a function to the contract that they call to execute arbitrary code. This is an issue for upgradeable contracts in general and is not specific to diamonds.
Well, exactly. How is this standard any different from the centralized owned upgradeable smart contracts out there? Why not a standard that abstracts upgrades being opt-in only by default?
This is great.
Would be curious to see a uniswap build using the Diamond pattern.
To me Uniswap is great exactly because it's not upgradeable/centralized.
@leonardoalt It is different in a lot of ways, but it is not different in regards to ownership/authentication. Ownership/authentication is not part of this standard. The ownership/authentication can be implemented in a diamond any way anybody wants.
I realize now that the way EIP-2535 Diamonds is currently written it misleads people into thinking that EIP-2535 Diamonds specifies how authentication/ownership is implemented or should work. It doesn't. So I think I will change the text you quoted because it is misleading. It certainly does not have to be that way, it just could be that way, depending on how ownership/authentication is implemented.
But note that the standard does suggest a different way to do ownership or authentication. See the "Decentralized Authority" section.
Also, the upgrade functionality can be removed in a diamond making a diamond immutable. This could be done by removing the cut function. And it possibly could be added back depending on certain cases dictated by the implementation of the diamond -- there are so many possibilities to what could be implemented and EIP-2535 Diamonds does not limit what can be done.
Why not a standard that abstracts upgrades being opt-in only by default?
Standards or tutorials or implementations that build on top of EIP-2535 Diamonds to provide different ownership/authentication/upgrade schemes are very much wanted!
EIP-2535 Diamonds provides basic architecture and structure and points of interoperability with user interfaces and software, and it leaves up to the implementer what the diamond does and how it works.
I don't know why you think a diamond can't be controlled by a single source.
The only reason I say this is because the single source would just publish documentation whenever they made a change. There's no real reason to emitt changes as events or capture this on-chain. Maker and other protocols already have governance processes to document this already.
A big use case is contracts with designs that exceed the max size of contracts, since diamonds don't have a max size.
That's a very good feature!
Companies are already using this kind of architecture (ERC1538) for their contracts, such as Caesar's Triumph, Enjin and others. It is real and in use today
I'll have to look more closely at these. This could be really useful for @austintgriffith DAOG game where the rules of the game are updated on-the-fly. Right now the rules are limited in scope, but this could allow more open-ended rules to be added or removed.
Permissions and authentication for cuts and access to storage variables can be handled in the same way as any other contract.
I'm glad you think this is possible because I really think this is the best potential use-case for this system. For instance, in the DAOG game, you could allow players to add/remove rules, without letting them interfere with the core game logic (i.e. the logic and storage deciding who wins and can withdraw the prize pot). Or with a Moloch, you could set certain thresholds for high/low impact changes to the dao contract itself. Dao members could pass smaller rules more easily, to run segregated and quick experiments, without risking the dao's funds.
the upgrade functionality can be removed in a diamond making a diamond immutable.
The upgrade functionality can also be removed by the contract not being upgradeable in the first place.
there are so many possibilities to what could be implemented and the diamond standard does not limit what can be done
Exactly my point. It is basically a really complicated way to make contracts as mutable as simply delegating everything to a dynamic address, which is actually a lot more transparent and simpler to read and check what it's doing. Complicated upgrade standards enable backdoors by obfuscation, especially when you can literally change everything.
@leonardoalt Its is really not very complicated. It is new and the standard does provide a lot of information to help use and implement the standard.
EIP-2535 Diamonds is this
- One function for adding/replacing/removing functions.
- One event for logging changes.
- Some functions for showing what functions the contract has.
A big part of the standard is transparency (logging changes), which removes obfuscation. I plan to make a user interface that shows and visualizes all changes to a diamond.
Some good discussion of EIP-2535 Diamonds here: https://ethereum-magicians.org/t/diamond-contract-standard/4038
Awesome work @mudgen!! I like this new version of the EIP, and have a few comments:
-
I really like the fact that
cutcan modify several functions atomically, and you don't need multiple calls, potentially leaving your contract in an inconsistent state. That said, I'd look into simplifying it a bit: instead of having the concept of add/remove/replace action, why not just havecutaccept an array of pairsselector,implementation, directly setting the implementation for each selector? Add and replace would be similar, and removal can be done by setting implementation to zero. Function clashes can be managed via appropriate tooling here. -
In line with the previous one, I'd also simplify the event. Having a big event with a bunch of actions is harder to consume. I'd just set up events with the
selectorandimplementationarguments, and fire as many as needed. You could even make selector indexed, so clients can monitor for changes to a specific function (I'd look into gas tradeoffs here though). -
There is another potential clash that is event clashing. The
DiamondCutevent could have the same selector as an event from a facet, potentially fooling a client who is listening to these events. That said, a client could just query the contract to verify the facet address if needed. -
I'd also simplify the loupe interface. I'd remove any enumerable features, and just keep a
getFacet(bytes4 selector)method. I'm not sure in which situations another contract would need to iterate over the facets of a diamond, and an offchain client could look into past events. I think this should also simplify the storage structures needed to support a diamond, making it overall more gas efficient.
@spalladino Thank you very much. This is great feedback! I appreciate that you take the time to look at this and write this feedback.
Awesome work @mudgen!! I like this new version of the EIP, and have a few comments:
- I really like the fact that
cutcan modify several functions atomically, and you don't need multiple calls, potentially leaving your contract in an inconsistent state. That said, I'd look into simplifying it a bit: instead of having the concept of add/remove/replace action, why not just havecutaccept an array of pairsselector,implementation, directly setting the implementation for each selector? Add and replace would be similar, and removal can be done by setting implementation to zero. Function clashes can be managed via appropriate tooling here.
I like the idea of simplifying the argument to cut. Your suggestion omits the string message part of the argument that describes a set of changes. It is a commit message. Removing it would simplify the argument to cut and reduce some gas. Do you think its disadvantages outweigh its benefits? The argument is as complicated as it currently is because it gives the ability to have different change messages for different sets of changes in one transaction. This helps document the changes, but maybe it isn't worth the added complexity of the data structure.
- In line with the previous one, I'd also simplify the event. Having a big event with a bunch of actions is harder to consume. I'd just set up events with the
selectorandimplementationarguments, and fire as many as needed. You could even make selector indexed, so clients can monitor for changes to a specific function (I'd look into gas tradeoffs here though).
The argument to the cut function and the argument to the DiamondCuts event is exactly the same. The cut function literally passes its argument directly to the DiamondCuts event without any manipulation. This is a nice simplicity. You are right, it would be nicer if it was a simpler data structure. It associates the change messages with their changes. Again, if we remove the string message then this data structure could be simplified. So I guess the important question is how important is it to have descriptive text describing changes to a diamond emitted in events? I suppose the datastructure could also be simplified some by only allowing one change message for all changes in one transaction. With the way it is now you can have multiple change messages for different sets of changes in one transaction. That is why the data structure is as complicated as it is.
- There is another potential clash that is event clashing. The
DiamondCutevent could have the same selector as an event from a facet, potentially fooling a client who is listening to these events. That said, a client could just query the contract to verify the facet address if needed.
Can you explain this more? I don't understand the scenario you are describing here.
- I'd also simplify the loupe interface. I'd remove any enumerable features, and just keep a
getFacet(bytes4 selector)method. I'm not sure in which situations another contract would need to iterate over the facets of a diamond, and an offchain client could look into past events. I think this should also simplify the storage structures needed to support a diamond, making it overall more gas efficient.
Wow, that is an interesting idea. I see what you are saying, the loupe isn't needed if software can simply look at the events to determine which functions exist. I'm not sure why getFacet(bytes4 selector) would be needed by the standard. The main purpose of the loupe is to provide interoperability with user interface software and tooling, but if events already give that then they could be used. People can always add in their own functions like getFacet(bytes4 selector) for their own purposes. I am considering removing the loupe completely from the standard.
@spalladino What do you think now?
The commit message is definitely interesting, but shouldn't it be handled off chain? Perhaps the contract should just have a small identifier that points to an actual commit or release off-chain, for traceability from the source to the deployment?
Still, one option (following the address,selector pairs approach) would be for the cut event to publish the array of address,selector pairs, and the commit message, similar to what you originally proposed - though easier to parse for the client. You lose indexability of selectors, but it may not be too critical.
Can you explain this more? I don't understand the scenario you are describing here.
Sorry, forget about this one. I mixed up event topics and function selectors. The first topic for an event, which is derived from a hash of its name and args and identifies the type of event, is 32 bytes long -not 4 bytes like a function selector. So clashes between event names are not possible.
People can always add in their own functions like getFacet(bytes4 selector) for their own purposes. I am considering removing the loupe completely from the standard.
Maybe there is an opportunity to have this standard be automatically ERC165 compliant...? Haven't looked at it in-depth.
The commit message is definitely interesting, but shouldn't it be handled off chain? Perhaps the contract should just have a small identifier that points to an actual commit or release off-chain, for traceability from the source to the deployment?
Have the identifier where? The commit messages are not stored in the contracts, they are just emitted with the event that shows the changes. I'd rather just emit the commit message in the event than emit a hash of a commit stored somewhere because I don't think this would be useful for user interfaces that show people all the changes to a diamond, but the commit messages describing the changes could be useful. The idea of the user interface is that it would pull all the verified source code from somewhere like etherscan so people could easily see all the source code of all the facets used by a diamond, and in addition people could see the verified source code of how a diamond was in the past if it was cut. And people would see the commit messages describing the upgrades, why they were done etc.
Maybe there is an opportunity to have this standard be automatically ERC165 compliant...? Haven't looked at it in-depth.
I don't see how it could be automatically compliant. I do like ERC165 and I think it is good for people to use it.
I recommend that this standard use ERC-165, just for the cut function. This will help tools, such as Etherscan, recognize compliant contracts.
Because this contract is VERY general in purpose, the cut function should be renamed to a more qualified name such as diamondCut. Also I would like to see a discussion of the proposed name and any potential function selector conflicts in the rational section.
The event has arrays in it, so this limits the ability to search logs.
The API is overly complex:
enum CutAction {Add, Replace, Remove}
struct FacetCut {
address facet;
CutAction action;
bytes4[] functionSelectors;
}
struct DiamondCut {
FacetCut[] facetCuts;
string message;
}
interface Diamond {
function cut(DiamondCut[] calldata _diamondCuts) external;
event DiamondCuts(DiamondCut[] _diamondCuts);
}
It can be:
struct DiamondBatchCuts {
bytes4[] functionSelectors;
address[] implementations;
}
interface Diamond {
function diamondCut(DiamondBatchCuts calldata diamondCuts) external;
event DiamondCut(bytes4 functionSelector, address oldImplementation, address newImplementation);
}It is unnecessary to categorize adding, changing and removing. Simply, a zero address corresponds to no implementation and a non-zero address is an implementation.
Use extra bytes.
Nobody asked for this feature but I'll suggest it any way.
I assume the standard contract is implemented like:
contract DiamondImplementation {
mapping (bytes4 => address) implementations;
function diamondCut(DiamondBatchCuts calldata diamondCuts) external {
for (uint i = 1; i < diamondCuts.functionSelectors.length, I++) {
address old = diamondCuts.functionSelectors[I];
implementations[diamondCuts.functionSelectors[i]] = diamondCuts.implementations[i];
emit DiamondCut(diamondCuts.functionSelectors[I], old, diamondCuts.implementations[I]);
}
}
}So this means the storage is mapping (bytes4 => address) implementations;. That is wasteful because you are only putting 160 bits in a 256 bit register.
You can store more... but what? The selector in the target contract!
So you can have a function selector a on your Diamond contract be implemented by function b on the implementation contract. Why would you want to do that? Because of code reuse.
OR you can ignore this suggestion entirely if all the implementation contracts are implemented using the fallback function, which is better.
E.g. specify that all implementations are:
interface DiamondFunctionImplementation {
fallback () external {
// code goes here
}
}^^ this will be more efficient.
I didn't actually read the EIP some maybe you already specified this.
Documentation on storage mutability is insufficient. This is a major design consideration. And as somebody that audits contracts I'll hate auditing this kind of contract :-~~~
Mutability is bad. As stated before in my prior related review. I'm still not a fan of using this EIP or Zeppelin OS for upgradeable contracts. If you want to upgrade your contract then best practice is to deploy a new contract and spend your marketing budget to inform everybody of the new version.
This is what I did, multiple times, when working on ERC-721. Since I published the first "ERC-721 compliant contract" (it's Su Squares, check it out) that means I needed to redeploy it every time there was a new ERC-721 draft. That's okay, and all of the wallet providers know me because I kept having to bother them to update the contract address in MetaMask, MyEtherWallet, etc. And that's a good thing.
An exception is zero-knowledge proof contracts. These require an enormous amount of code. And these require a limited caveat to my note above. It might be reasonable for a ZKP contract to be deployed in multiple stages. But the contract should not be open to the public until deployment is completed (dependent contracts are loaded) and no further changes should be possible after deployment. Even still, the functionality of Diamond contracts should not be necessary for this deployment strategy.
@fulldecent I appreciate your feedback on this.
I recommend that this standard use ERC-165, just for the cut function. This will help tools, such as Etherscan, recognize compliant contracts.
Yes, I'll add this to EIP-2535 Diamonds.
Because this contract is VERY general in purpose, the cut function should be renamed to a more qualified name such as diamondCut.
I agree that cut should be renamed to a more qualified name. I'll rename it to diamondCut.
Also I would like to see a discussion of the proposed name and any potential function selector conflicts in the rational section.
There is a section about function selector conflicts in the security section.
The API is overly complex:
The argument to cut is complex because it supports descriptive messages describing changes, like commit messages. And different messages can be associated with different sets of changes. The argument can be simplified if we throw out the descriptive messages of the changes, but I think it is worth it to keep them. I understand that documentation messages are not sufficient for a security audit but I think they can help people understand diamonds and why/what changes are made.
An important part of EIP-2535 Diamonds is creating user interfaces that pull all the verified source code that is used and displaying it in such a way that a person can see and understand all the code that is currently used by a diamond and also look at past code that was used before it was cut.
It is unnecessary to categorize adding, changing and removing. Simply, a zero address corresponds to no implementation and a non-zero address is an implementation.
It is necessary to prevent function selector conflicts. That's why it is there. The alternative is to let the user or off-chain software first verify that they aren't making any function selector clashes before calling cut but a diamond can't enforce they do this and it may be easier for them to specify add,replace,remove and the diamond can enforce they do this.
contract DiamondImplementation {
I understand your DiamondImplementation and I understand that an address is 160 bits and that additional data can be stored in the 256 slot. But after that I am lost. But I am interested. Can you explain more?
As stated before in my prior related review.
I think I kinda remember your prior review but for some reason I can't find it. Do you happen to know where it is? Because I'd like to review it if we can find it. Nevermind I found it! It was an email to me and I found it.
I remember you working and handling Su Squares and I thought you did a good job with it and the way you did things with it was good.
This change is complete: I renamed the cut function to diamondCut in the standard.
Wow, that is an interesting idea. I see what you are saying, the loupe isn't needed if software can simply look at the events to determine which functions exist. I'm not sure why getFacet(bytes4 selector) would be needed by the standard. The main purpose of the loupe is to provide interoperability with user interface software and tooling, but if events already give that then they could be used. People can always add in their own functions like getFacet(bytes4 selector) for their own purposes. I am considering removing the loupe completely from the standard.
Okay, here's the truth, I just don't totally trust 100 percent that events will be available all the time, forever, and with good performance all the time, forever. Maybe that is dead wrong and hope it is wrong and I'd love someone to prove it to me that it is wrong so I am totally convinced. I want to be overly safe until then -- after all we are dealing with diamonds!. Having the loupe functions implemented is a very good guarantee that you will be able to inspect your diamonds for facets and functions. If it is implemented right and it doesn't work then that means ethereum contracts don't work anymore and we have bigger problems.
So I'm keeping the loupe functions in the standard. Actually I removed two of them: functionSelectorByIndex and totalFunctions, because I don't see the need for them with the other ones. Also, by having events and loupe functions it provides two different ways to get the functions and facets of a diamond. If one way breaks for some impossible reason, the other way is available.
I am going to support @leonardoalt fully here. Rare are the occasions where upgradeability on-chain cannot be replaced with off-chain mechanisms to achieve the same end result.
As an added problem, the more you complicate on-chain upgradeability mechanisms, the more obfuscated and less auditable these become. This means that clients' trust on the system is greatly reduced.
If everything is mutable why not just delegate execution?
I am going to support @leonardoalt fully here. Rare are the occasions where upgradeability on-chain cannot be replaced with off-chain mechanisms to achieve the same end result.
As an added problem, the more you complicate on-chain upgradeability mechanisms, the more obfuscated and less auditable these become. This means that clients' trust on the system is greatly reduced.
If everything is mutable why not just delegate execution?
I feel like this approach takes the stance that we're never going to improve the way we do things today.
The audit log itself ensures that no central party has to prove what the state is, as it's self managed.
As a user if you want any real trust with proxies, you want to have your own, anytime an app needs a proxy they deploy one for you, as a user, you could have A proxy that you trust, that you cut with any features that you need, without having to extend that trust to anyone else to verify what the state of your proxy is.
Governance maintained Diamonds with the ability to cut in new features seems like a huge boon in terms of how we manage and maintain upgradeability.
No question there are new security challenges to deal with, but these types of patterns work well in other application development to date.
As a user you should be asked to opt-in an upgrade, not be forced to trust obfuscated code.
As a user you should be asked to opt-in an upgrade, not be forced to trust obfuscated code.
No disagreements there, which is why its great for user owned contracts.
@leonardoalt @androolloyd @GNSPS One thing to keep in mind, which @fulldecent pointed out, is that this standard is very general, which also means that it is extremely flexible. It is easy to make the error of pegging this standard to a particular use case or to make assumptions about it.
As a user you should be asked to opt-in an upgrade, not be forced to trust obfuscated code.
@leonardoalt This standard is probably flexible enough to accommodate that. I'm interested in more details about how that could work.
I myself am very guilty of pegging this standard to a particular use case: "upgradeable contracts". This standard is useful for creating very useful immutable contracts that can't be upgraded. How so? Well there might be many ways (being so general and all) but I think of two really good use cases. But before I tell you the use cases let me tell you how to make useful immutable contracts with this standard.
The standard has been carefully edited to say that a diamond "uses" the diamondCut function. It does not say that the diamondCut function has to be added to the diamond as one of its functions. So this means that you can delegatecall to the diamondCut function in the constructor of the diamond to add all the functions needed by the diamond. The example in the standard shows how to call diamondCut in the constructor of a diamond. Since the diamondCut function is never actually added to the diamond the diamond is immutable with the same immutability and trust guarantees as a vanilla contract. My two use cases illustrate why this is useful:
-
If you have a contract that hits the max contract size limit and there is too much code or dependency upon storage variables to separate it out into regular contracts, then you can make it an immutable diamond (which has no max size). Cool thing about making it a diamond is that you still break your big contract into smaller contracts, modularizing your code to a degree. A good description of a diamond is this: A group of contracts that share the same storage variables and address.
-
You can start with an upgradeable diamond in your development and testing and upgrade it to your heart's delight. Reap the advantages of easy upgrading and a stable address as you work out new features, bugs and kinks. Release the upgradeable contract on a test network with your application for beta testing and upgrade it when needed. This is iterative development. When it is all solid then make it an immutable diamond and launch it on the main network.
Obfuscation exists when there are no tools to make something transparent and clear. This standard standardizes diamonds so that tools can be written for them so they are transparent and clear.
@androolloyd I love this use case:
As a user if you want any real trust with proxies, you want to have your own, anytime an app needs a proxy they deploy one for you, as a user, you could have A proxy that you trust, that you cut with any features that you need, without having to extend that trust to anyone else to verify what the state of your proxy is.
I want everyone to have their own diamond.
The only reason I say this is because the single source would just publish documentation whenever they made a change. There's no real reason to emitt changes as events or capture this on-chain. Maker and other protocols already have governance processes to document this already.
@pi0neerpat I realized that you are right. It is better to let people document their contracts and their upgrades off chain the way they want to. Thank you for this useful feedback. I updated the standard to reflect this. Specifically I removed the documentation message part of the input and event.
The commit message is definitely interesting, but shouldn't it be handled off chain? Perhaps the contract should just have a small identifier that points to an actual commit or release off-chain, for traceability from the source to the deployment?
@spalladino I realized that you are right. I removed the commit message functionality. I think that the facet addresses can be used as pointers to facet and upgrade documentation, in the same way they are used as pointers to verified source code on etherscan. Thank you for this very useful feedback.
I really like the fact that cut can modify several functions atomically, and you don't need multiple calls, potentially leaving your contract in an inconsistent state. That said, I'd look into simplifying it a bit: instead of having the concept of add/remove/replace action, why not just have cut accept an array of pairs selector,implementation, directly setting the implementation for each selector? Add and replace would be similar, and removal can be done by setting implementation to zero. Function clashes can be managed via appropriate tooling here.
@spalladino You are right about simplifying the argument to the diamondCut function. After removing the message part I was able to simplify it and I updated the standard.
The add/remove/place action is used to prevent function selector clashes. I don't feel comfortable with delegating that responsibility to tools because some tools might not do it. Also, having the action makes explicit what is happening which is good. I know it is redundant with events but someone can look at past transactions and know whether functions were added or replaced.
The event has arrays in it, so this limits the ability to search logs.
@fulldecent You are right about the events. I changed the events in the standard to something very similar to what you suggested.
In line with the previous one, I'd also simplify the event. Having a big event with a bunch of actions is harder to consume. I'd just set up events with the selector and implementation arguments, and fire as many as needed. You could even make selector indexed, so clients can monitor for changes to a specific function (I'd look into gas tradeoffs here though).
@spalladino You are right about events. I updated the standard with events like you suggested. I am leery about making the function selector indexed because of the extra gas cost and how useful it might be.
The add/remove/place action is used to prevent function selector clashes. I don't feel comfortable with delegating that responsibility to tools because some tools might not do it. Also, having the action makes explicit what is happening which is good. I know it is redundant with events but someone can look at past transactions and know whether functions were added or replaced.
Not sure if this will help, but I made a tool to assist in stripping out variables for both turning a regular contract into a proxy, and to help when performing an upgrade. It gathers all variables from all contracts (so dev can write new code as normal, without thinking of proxy) and throws them into Storage.sol. The dev can then just git diff to make sure variables are appendend and not reorderes. In this case, it's the EIP1822 Universal Upgradeable Proxy Standard.
@pi0neerpat sol-proxy is interesting, thank you.
How can we isolate storage on the same permissioned system that we use to allow cuts?
@pi0neerpat I am glad you ask this because it is a very good question. I've thought about it a lot. The answer is we can't. If a person can add/replace functions then he/she can alter storage willy nilly. But the story does not end there. There are things that can be done to eliminate, reduce or limit the danger to storage. I just added a new subsection to the standard called, "Security of Diamond Storage" to address this.
Love this. One quick comment: I don't see any issue with leaving the upgrade message string in the event. If worried about gas, simply leave it blank (I did this many times when upgrading my ERC-1538 functions). Reasons you may want the message logged on chain is if you are upgrading/cutting a number of functions in different contracts one at a time over months. Maybe you are fixing a minor bug in a function and forget to comment the in the function header the reason for the upgrade, or maybe from an operational perspective you may not remember to log all new code updates to etherscan for example. Thus the upgrade message may become useful tool for documenting the reason for the upgrade (and again, you have the option to leave the message blank if you choose).
Note: @Droopy78 is an experienced developer with ERC1538 in production.
@Droopy78 yes, I hear you. I have gone back and forth about it a lot. I also developed ERC1538 contracts and I liked having the commit messages. I take what you say with strong consideration.
An earlier version of the diamond standard had the message string. Here's what it looked like:
struct FacetCut {
address facet;
CutAction action;
bytes4[] functionSelectors;
}
struct DiamondCut {
FacetCut[] facetCuts;
string message;
}
interface Diamond {
event DiamondCuts(DiamondCut[] _diamondCuts);
function cut(DiamondCut[] calldata _diamondCuts) external;
}This was really nice because it enabled you to put in documentation messages in multiple places within the cut.
The diamond standard is different than ERC1538 in that the updateContract function can only work with one delegate contract at a time, but the diamondCut can work with multiple facets at a time. I've found that adding the message string to the diamondCut argument in a good way complicates the argument. But this in itself is not a good reason not to do it.
@spalladino and @pi0neerpat have mentioned that documentation of upgrades could be handled off chain. I've really considered each side of this. My personal opinion is that contract documentation and documentation of upgrades should be done off chain and should be editable so it can be added and improved over time. Just like there is an etherscan API for retrieving the verified source code and ABI of a contract I think there should be an API for retrieving the documentation, including upgrade documentation for contracts and diamonds. And the documentation should be able to be edited so it can be improved and updated, not fixed in time.
I don't want this standard to enforce people to do something that isn't absolutely necessary for interoperability with tools. At the same I don't want the standard to stop people from having things the way they want them. So I'm making the standard give what I think is absolutely necessary to be standardized but at the same time giving the freedom to do more and enable people to make things the way they want them.
Freedoms of note:
- The standard does not prevent people from adding and using their own custom functions for adding/replacing/removing functions from a diamond. They should still provide and implement the standard
diamondCutfunction if their diamond is not immutable for interchangeability reasons if the person ever wants their diamond to work with other tools that add/replace/remove functions. - People can emit other events in the implementation of the
diamondCutfunction and add whatever custom functionality they want to it.
@mudgen My comment was making the assumption that each update (even one that contains multiple diamandCuts across multiple contracts) should be in relation to only one change, and thus only one commit message would be needed per update/commit (instead of one per diamondCut, which I agree is too complicated). It is similar to committing changes to github. Good practice is to commit a group of files that relate to one feature or bug fix at a time. Yes, you can commit 5 unrelated features or bug fixes with one commit, but it's tough to unravel later if you have to roll back one of those changes.
That being said, I am ok with the decision to leave the commit message out of the standard.
@Droopy78 Yes, that make sense. I see what you mean.
The event has arrays in it, so this limits the ability to search logs.
@fulldecent How so? Please explain.
The event has arrays in it, so this limits the ability to search logs.
@fulldecent How so? Please explain.
You can only filter by indexed event topics. You can index up to 4 event topics, and arrays are not indexable. Usually the event type is the first (0th) topic.
@wjmelements Thanks, that is what I was thinking but wanted to make sure that @fulldecent wasn't referring to something else.
I'm not certain about this, but I believe the Argent wallet has taken a modular approach to upgrades, where a small piece of the wallet functionality can be upgraded while leaving most of the contracts untouched. In addition, wallet users need to opt in to use the upgraded portions of code.
While this is probably out of scope for this standard, it does demonstrate a way to make use of this type of architecture, while providing a guarantee to users that things won't change unexpectedly.
@nfurfaro I think that is a great example.
Somebody asked me if they should use the Diamond Standard or ERC1538. Here is my answer:
Definitely go with the Diamond Standard. Here are some reasons why:
The function that is used by the Diamond Standard to add/replace/remove functions is more capable than in ERC1538. In the Diamond Standard this function, called
diamondCut, can add/replace/remove multiple functions from multiple facets in one transaction. This ability enables you to make all changes necessary without your diamond being left in a temporary incomplete state. In contrast the same function in ERC1538 can add/replace/remove multiple functions from only one facet in one transaction.The
diamondCutfunction uses less gas than the equivalent function in ERC1538.More tools and support can be expected for the Diamond Standard. For example there is a plan to add special deployment support to Truffle for diamonds: https://twitter.com/mudgen/status/1246991530478501889 My personal marketing, support and work is with the Diamond Standard. I consider ERC1538 deprecated. The author of ERC1538 agrees with this.
ERC1538 was and is great. Lessons learned from it and the improvements in the Solidity language have been applied to the Diamond Standard.
That is awesome what you are doing. I can definitely see a diamond as the glue that connects everything together.
Just implemented this for our project and it went quite well -> https://hiddentao.com/archives/2020/05/28/upgradeable-smart-contracts-using-diamond-standard
One suggestion I'd like to make is that instead of DiamondExample manually configuring the method selectors, have an interface for fetching method selector info from facets such that facets themselves tell the proxy what methods they expose (thereby reducing coupling between these components):
interface IDiamondFacet {
function getSelectors () external pure returns (bytes memory);
} Example facet using it:
contract TestEntityFacet is IDiamondFacet {
function getSelectors () public pure override returns (bytes memory) {
return abi.encodePacked(
TestEntityFacet.getNumPolicies.selector
);
}
function getNumPolicies() public view override returns (uint256) {
return 666;
}
}Then you could have a method which iterates through passed-in facets to aggregate this info and pass it onto diamondCut():
function _registerFacets (address[] memory _facets) internal {
bytes[] memory changes = new bytes[](_facets.length);
for (uint i = 0; i < _facets.length; i += 1) {
IDiamondFacet f = IDiamondFacet(_facets[i]);
bytes memory selectors = f.getSelectors();
changes[i] = abi.encodePacked(_facets[i], selectors);
}
bytes memory cutFunction = abi.encodeWithSelector(Diamond.diamondCut.selector, diamondCut);
(bool success,) = address(diamondFacet).delegatecall(cutFunction);
require(success, "Adding functions failed.");
}
}As outlined in the blog post we also had to make adjustments to how diamondCut() gets called in order to control access to the upgrade mechanism.
Perhaps the above ideas can be incorporated into the spec to make implementing the standard easier for people?
I had this sitting around for some reason...
- Here's an interface:
interface ERC2535 /* is ERC165 */ {
struct implementation {
bytes4 selector;
address target;
}
// Emitted for each changed implementation
event DiamondCut(bytes4 functionSelector, implementation oldImplementation, implementation newImplementation);
// Message is not in the event. Intentional. You could look it up in the chain data if needed. Expected not needed.
}- Here is THE implementation:
contract Diamond is ERC2535 {
struct implementation {
bytes4 selector;
address target;
}
mapping (bytes4 => implementation) functionImplementations;
address reserved;
fallback() external payable {
require(functionImplementations[msg.sig], "Function is not implemented");
assembly {
//TODO: Call the function...
switch result
case 0 {revert(ptr, size)}
default {return (ptr, size)}
}
}
function diamondCut(
bytes4[] functionSelectors,
implementation[] oldImplementations,
implementation[] newImplementations
string calldata message
) internal {
for (uint i = 1; i < functionSelectors.length, I++) {
bytes4 selector = functionSelectors[i];
implementation old = oldImplementations[i];
implementation new = newImplementations[i];
require(old == functionImplementations[selector]);
functionImplementations[selector] = new;
emit DiamondCut(selector, old, new);
}
}
}@fulldecent Thanks for sharing that. Your implementation is the basic idea of the standard.
This standard is moving along. It was recently popular on Reddit: https://www.reddit.com/r/ethereum/comments/gze6k3/a_diamond_is_a_set_of_contracts_that_can_access/
Some people are using the standard and its precursor (ERC1538) and some are expressing interest in using it. Today a new Python tool for automating the upgrade process for diamonds was released. Here is an article about that: http://joeyzacherl.com/2020/06/diamond-setter-ethereum-smart-contract-manager/
And here is the tool: https://github.com/lampshade9909/DiamondSetter
Here's a recent article about the Diamond Standard by someone who is using it in production: https://hiddentao.com/archives/2020/05/28/upgradeable-smart-contracts-using-diamond-standard
I understand that marketing can be important, but I feel it's in poor taste to open near-identical issues on so many repos in such a short period. This is the first time I've seen what feels like spam on Github. I think you might be doing yourself a disservice here.
@iamdefinitelyahuman You are right, I am sorry about this. Any ideas on what I should do to fix this and find a better way to let tools know about the Diamond Standard?
Here is how to properly "spam" other projects to adopt a standard:
- Identify projects that might benefit from your standard
- Pay somebody to implement on each of those projects
- Submit and PRs and start discussion
Do things that don't scale...
That makes sense. Thanks @fulldecent.
Rare are the occasions where upgradeability on-chain cannot be replaced with off-chain mechanisms to achieve the same end result.
As an added problem, the more you complicate on-chain upgradeability mechanisms, the more obfuscated and less auditable these become. This means that clients' trust on the system is greatly reduced.
If everything is mutable why not just delegate execution?
@GNSPS
The Diamond Standard has been modified and improved since you first wrote about it. Please take another look.
Diamonds are not just an upgrade mechanism. They are a way to organize Solidity code to achieve cohesion and modularity. Check out my new article about diamonds: https://dev.to/mudgen/understanding-diamonds-on-ethereum-1fb
Diamonds don't have to be upgradeable, or the upgrade capability can be removed. The Single Cut Diamond pattern that is written about in the standard is a nice way to do iterative contract development and/or testing on a test network. Do all your upgrades during development and testing and then deploy an immutable Single Cut Diamond on mainnet. It has the same immutability guarantees as a regular contract.
Diamonds are flexible and capable when it comes to upgrading them. But the DiamondCut event tracks and records every upgrade. So when you are auditing a diamond you can use this list of events to see what has changed and when it has changed.
I think that how much work it takes to audit an upgradable contract is related to how much is changed over time. If many things are changed then there is more work to see those changes and how they have affected contract storage.
So a diamond could be easy to audit if it hasn't changed much or more difficult to audit if it has changed a lot. But this is the same for any kind of upgradeable contract.
But with diamonds you are working with a proxy contract that has its contract storage modified by multiple facets. Each facet has its own source code. So when auditing a diamond you have to see how the source code of the different facets affects the contract storage of the proxy and you have to see how the different facets might interact with each other via the proxy contract storage. I agree this is more complicated to audit than a single contract. With more flexibility more complexity is possible. There are tradeoffs here.
If I have assimilated this correctly, the diamond standard implements most of the following model of a router with 2 contracts.
Each contract may have its own data (i.e. DataR, Data1, Data2) and share data with other contracts (i.e. Data12) or with other contracts and the router (DataX).
Any shared data can be located in memory using the mapping of a hash technique which is imported into all the using contracts.
This is very much like the memory model of an embedded system with a scheduler and 2 processes.
Data12 would contain the inter-process queues.
DataX would contain system gauges/whiteboards and logging.
Data1 and Data2 would contain contract specific data.
I think that the similarity effectively validates your design pattern.
I see the big difference being the DiamondCut function you are proposing, which is an implementation choice.
@JulesGoddard Yes, this is a fantastic perspective and example of a diamond.
Curious, did you make that diagram or is that part of an existing system?
The purpose of the diamondCut function is to provide compatibility among tools that help make and form diamonds.
@mudgen Yes, I drew that diagram. You can just make out the datona logo in the background.
It is based on many similar diagrams I have drawn in the past for embedded system design.
But I was actually prompted by your diagram in https://dev.to/mudgen/understanding-diamonds-on-ethereum-1fb
That is very similar, it's just missing the diamond data (DataR in my diagram) and overall common data (DataX).
@JulesGoddard Very cool.
👏 @JulesGoddard great diagram thanks 📖
I want to add new functions to the Diamond Standard reference implementation. If you want to contribute to an interesting project, here's your chance. See this issue: https://github.com/mudgen/Diamond/issues/7
I think that the Diamond Standard EIP should identify the function of Diamond Storage much earlier in the description.
In my opinion, this should be a fundamental part of Diamond Standard and should be introduced in the simple summary, abstract and motivation.
A group of facets (a side of a diamond?) can use data that is common to them but not available to the master diamond contract or to other facets.
@JulesGoddard You are right. Thanks for reminding me about this. The reason Diamond Storage isn't as much a focus in the Diamond Standard is because the Diamond Standard was written before Diamond Storage existed, but that's no reason for the Diamond Standard not to be updated.
I plan to update the Diamond Standard EIP to give Diamond Storage a more central focus.
@mudgen I realised that. However, my concern is that without the common data/Diamond Storage feature, all contracts are only able to share the same data, and that is less likely to be a useful pattern. I think that the more generic, and probably much more useable, pattern is where groups of facets may use common data/Diamond Storage. That is why I was suggesting you reorganise your EIP (if you are permitted to) and accentuate the common data/Diamond Storage feature.
@mudgen Please consider this:
Simple Summary
A diamond is a set of contracts that can access the same storage variables and share the same Ethereum address.
cf
A diamond is a set of contracts that share the same Ethereum address and can access common storage variables as well as their own.
Abstract
- ... 6.
- Facets can use their own data, as well as data common to multiple facets or to the whole diamond.
Why Make a Diamond?
- ... 6.
- You develop features independently, which may need to communicate with other features. Use storage which is common between the contracts.
Etcetera.
@JulesGoddard Thanks. I'm working on it slowly. I just added this description to the Abstract:
A way to create a set of contracts (facets) under a single address that are independent from each other but can share functionality and contract storage.
I'd like to figure out more ways for people to learn/understand diamonds and what they can do and achieve with them.
@mudgen Fair enough. I think that I know what you are intending to achieve.
The question is, how can we communicate the maximum amount of information with the minimum quantity of words? Very difficult. Especially when some words are particularly ambiguous or domain-specific.
In terms of Diamond Standard, it's reasonable to assume a fair knowledge of Ethereum: contracts have their own storage, libraries don't have storage.
Where you want to deviate from that norm, that's where you need to explain, in my humble opinion.
How can contracts 'share functionality?' Suggest worth explaining.
What about 'sharing contract storage?' Suggest introduce Diamond Storage (or common data) here.
The question is, how can we communicate the maximum amount of information with the minimum quantity of words? Very difficult. Especially when some words are particularly ambiguous or domain-specific.
In terms of Diamond Standard, it's reasonable to assume a fair knowledge of Ethereum: contracts have their own storage, libraries don't have storage.
Where you want to deviate from that norm, that's where you need to explain, in my humble opinion.
How can contracts 'share functionality?' Suggest worth explaining.
What about 'sharing contract storage?' Suggest introduce Diamond Storage (or common data) here.
Exactly! A challenge. Even if I rewrite the Diamond Standard in a good way it won't be enough.
This is why I need your help, to explain these things in other ways, to break things down further, to give relatable analogies like memory used in embedded systems, more diagrams, examples of diamonds and their explanation, ways diamonds can be used, tutorials, videos that explain and show diamonds. I can add a supplementary section at/near the end of the standard to link to articles and material that give information about diamonds.
There is something missing in the diamondCut function, which has been mentioned by @wighawag and @dOrgJelli. diamondCut has no way to execute arbitrary code in order to initialized contract storage for functions added/replaced/removed.
So I am thinking of changing diamondCut to give it another parameter: bytes memory _initialization which contains 0 bytes if there is no initialization and otherwise contains a contract address and additional bytes. The initialize(bytes calldata _data) function will be called with delegatecall on the contract address and the additional bytes will be passed in the _data argument.
Thoughts?
@androolloyd @pi0neerpat @leonardoalt @spalladino @fulldecent @GNSPS @Droopy78 @wjmelements @nfurfaro @hiddentao @sambacha
I want to thank you guys for your comments and feedback about the Diamond Standard.
The Diamond Standard has changed significantly since you first read it. The standard is different and much of the standard has been rewritten.
I'd really appreciate it if you would read the NEW Diamond Standard with fresh eyes here: https://eips.ethereum.org/EIPS/eip-2535
As always, your feedback is helpful and appreciated.
I have tried to make diamonds more useful and easier to understand. Let me know if I succeeded.
I wrote a blog post that describes changes to the Diamond Standard here: https://dev.to/mudgen/update-what-s-new-in-the-diamond-standard-eip-fjk
@mudgen Fresh eyes here. I really like the rewrite of the documentation. After first pass review here are some comments:
Upgrades and Immutability
I would separate out all three life-cycle cases for clarity. Suggested wording:
"A diamond can be immutable from inception by not adding any external functions that can add/replace/remove functions. The primary use case would be unlimited contract size.
A diamond that is mutable can be made immutable by removing such functions.
A diamond can be upgraded if has a diamondCut external function or other function(s) that can add/replace/remove functions."
Other comments
I like how you only bring up the facet and diamond terms in the first half of the write up, then wait until later to introduce loupe. Previously, I found it to be more confusing when all the terms were introduced at the onset.
In the "Facets and State Variables" section, it would be more clear to use the same naming in the example code as is used in the diagram in the following section showing DiamondStorage1, DiamondStorage2, etc, so that the reader can connect the code example with the diagram more easily (or maybe I didn't understand it on first pass).
General question: Are there specific scenarios where someone would want to create their own interface to add/remove/upgrade functions instead of using the DiamondCut function defined in the interface?
If there isn't a compelling case, I think it would make the standard and documentation more clear throughout to simply make DiamondCut mandatory instead of trying to be overly flexible. The add/remove/upgrade Event is already mandatory. It would make the Life Cycle descriptions I mentioned above more succinct as one example.
Also, as a counterpoint to the argument that DiamondCut is simply “used” and not mandatory in the immutable case: The constructor always needs DiamondCut even in the immutable diamond case. It is needed for the initial cut the diamond. Following that, yes it wouldn’t be added as a facet, but it was still a necessary function for the Diamond standard to work.
Thanks, and again great job on this!
Motivation and Abstract sections are missing, and there are a bunch of sections between the Simple Summary and the Specification that probably should be rolled up into one of the template sections.
I would really like to see a strong argument why this needs to be a standard rather than a best-practice or good-idea. In general, standards are useful when there is a many-to-many relationship between clients and providers. In this case, I do not see a clear many-to-many relationship.
As an example, object oriented programming, functional programming, and dependency injection are all design patterns/good ideas but none of them benefit from being standardized. On the other hand, browser plugins are standardized because there is a many-to-many relationship between browsers and extension developers.
I think I mentioned this somewhere else but I don't see it here, and figured it would be good to have a record of it. I recommend using a more intuitive naming convention rather than the clever "diamond, facet, loupe, etc." naming convention. New jargon should be introduced only when absolutely necessary as in raises the barrier to entry for new developers.
Thanks for the helpful feedback! Yea, your descriptions of upgrades/immutability are good. I added them to the standard.
Good idea about making the example code consistent with the diagrams. It's hard for me to do that for various reasons but I updated the standard with some code changes to make it more consistent. I didn't think of that, thank you.
The standard diamondCut function is general and designed for maximum flexibility so it can be used by all tools in lots of ways. People may want to have versions that are more specific. The diamond reference implementation uses a custom version of diamondCut in the constructor of the diamond to add functions. It is an internal function without the address _init, bytes calldata _calldata parameters.
@Droopy78 Thanks for your acknowledgement and help. I look forward to reading about your diamonds.
@MicahZoltu It is true that the Diamond Standard encapsulates some good ideas and a pattern for developing flexible smart contract-based applications. And the EIP does discuss the pattern a lot and how it works. I'm glad you brought up reasons for standardization because I think it deserves more attention. And it is an area that people and developers can really help.
Standardization is needed to make the diamond pattern more useful and fulfill its design.
Specifically standardization is needed for these two things:
- Transparency
- Interoperability
Transparency
A key feature of the Diamond Standard is transparent upgrades on blockchains. The Diamond Standard used to be the Transparent Contract Standard. There is an interesting article about it that covers transparency.
A diamond provides at least two kinds of transparency:
- Realtime transparency, using the diamond loupe.
- Historical transparency using the DiamondCut event.
The diamond loupe enables people to see what functions and facets a diamond currently has. This is standardized so that tools can be made to show people what functions and facets a diamond has. There is a many to many relationship here. There can be many diamonds. There can be many (or multiple) client software programs that show diamonds. Standardizing the loupe functions makes this many to many relationship possible. Standarizing the loup functions encourages software to be developed that makes diamonds more transparent.
The DiamondCut event is emitted any time any function is added/replaced/removed on a diamond. This is standardized so that multiple software programs can be developed that can show all upgrades to all diamonds on a network, thus making diamond upgrades transparent.
Etherscan has expressed some interest in supporting diamonds and I think there will be more interest in creating diamond transparency tools and user interfaces as more projects use the Diamond Standard. More projects are using the Diamond Standard. I know of several projects in development that are using diamonds.
Interoperability
The diamondCut function adds/replaces/removes any number of functions on a diamond in a single transaction. This function is standardized so that multiple tools can be made to create diamonds and upgrade upgradeable diamonds. This enables someone with diamonds to migrate or use different tools, or have more choice of tools, for upgrading or creating diamonds.
A couple tools have been made or added support for creating, deploying and upgrading diamonds:
I expect more tools to be created.
What is good about this standard:
- Works around contract size limit
- Comparable overhead to other SLOAD+DELEGATECALL upgradeable proxies.
- Standardization helps universal dapp services like etherscan expose the current ABI.
What is bad about it:
- Naming: diamonds are hard, proxies are soft
- Method lookup overhead in the top-level proxy
My preferred proxy pattern is the CREATE2 reincarnation, which has no proxy overhead, but doesn't handle the contract size limit.
I also suggest moving all diamond interface functions, including DiamondLoupe, to an initial facet, so that upgradeability can be upgraded or removed.
I also suggest finding a use for the top 12 bytes of the selectorToFacet mapping. It could be used to distinguish delegatecall from possible call or callcode facets, but there might be something even better. Otherwise it's fine to leave it unused because such bloat also impacts gas.
@wjmelements Thank you for this great feedback!
My preferred proxy pattern is the CREATE2 reincarnation, which has no proxy overhead, but doesn't handle the contract size limit.
This is interesting, thank you. I'd like to implement diamonds by storing the function selectors and facet addresses in the bytecode of the diamond using CREATE2 and then finding which facet to use in a fallback function, perhaps using a binary search on the function selectors. All to avoid the SLOAD but keep the diamond pattern. This is something I plan to work on.
I also suggest moving all diamond interface functions, including DiamondLoupe, to an initial facet, so that upgradeability can be upgraded or removed.
Yes, this is done in the diamond reference implementation.
I also suggest finding a use for the top 12 bytes of the selectorToFacet mapping. It could be used to distinguish delegatecall from possible call or callcode facets, but there might be something even better. Otherwise it's fine to leave it unused because such bloat also impacts gas.
Yes, the 12 bytes in the diamond reference implementation are used to efficiently store information needed to keep track of function selectors that are returned by the loupe functions.
Interesting idea to let the user choose which call opcode to use for an external function. I'll chew on that.
Checkout the diamond reference implementation: https://github.com/mudgen/Diamond
Thanks!
Hello all! I'm here from @trailofbits to let everyone know that we conducted a paid security review of this EIP for a client. Results are forthcoming, however, we felt the need to immediately advise others that they should avoid using this code.
Hello all! I'm here from @trailofbits to let everyone know that we conducted a paid security review of this EIP for a client. Results are forthcoming, however, we felt the need to immediately advise others that they should avoid using this code.
This is unsurprising since Trail of Bits strongly advises against using upgradeable contract patterns, and particularly proxy contract patterns as can be found in their article about upgradeable contracts.
Trail of Bits also advises others not to use OpenZeppelin's proxy contracts.
Right now when using a diamond from EIP-2535 Diamond Standard, and using the diamondCut function to remove functions from a diamond, should trying to remove a function that already does not exist cause the transaction to revert? Or should it be a noop and the transaction succeeds?
When trying to replace a function with the same function using the same facet, should the transaction succeed or should it be a noop and the transaction succeeds?
Upgrades should be done in a careful, intentional way, so it makes sense to me to revert in these cases. But if it does not revert then the diamondCut function is a little more flexible. What do you think?
Here's how it is currently written in EIP-2535 Diamond Standard:
The diamondCut function does nothing if a supplied function selector is already associated with the supplied facet address. The diamondCut function does nothing if a supplied function selector can’t be removed because it already does not exist.
I am thinking of changing this in the standard so that these cases revert. Please give me your feedback and reasoning.
I am thinking of changing this in the standard so that these cases revert. Please give me your feedback and reasoning.
Opposed because the suggested change would require an additional SLOAD on the prior value. If gas costs never changed you could use GAS to implement the behavior.
Another reason to oppose is that the diamondCut function loses idempotence.
Gas and idempotence are out of scope
The intended use case is not specified (missing in Simple Summary, and EIP is woefully failing to meet correct EIP format). But I understand that the use case is a human administrator manually performing diamond upgrades.
In this case one-time SLOAD gas costs and idempotence are not a concern for administrators occasionally upgrading contracts.
Idempotence is bad
See also the ERC-721 discussion that led to the requirement that all token transfers will also specify the current token owner. Theoretically, this information is redundant (you could query it with ownerOf(tokenID). And even in this case, with token transferring happening frequently, and being initiated by computers in batch rather than humans occasionally, our consensus still found that the safety afforded by requiring this extra comparison outweighed the cost.
Thanks @wjmelements !
The SLOAD on the prior value is done anyway. You are correct that with this change diamondCut loses its idempotence.
@fulldecent Thank you for your input! Fully noted. Safety is really important. And I'd rather have security auditors pointing out less things.
I'm also considering another change to the standard.
Currently the function signature for diamondCut is this:
struct Facet {
address facetAddress;
bytes4[] functionSelectors;
}
/// @notice Add/replace/remove any number of functions and optionally execute
/// a function with delegatecall
/// @param _diamondCut Contains the facet addresses and function selectors
/// @param _init The address of the contract or facet to execute _calldata
/// @param _calldata A function call, including function selector and arguments
/// _calldata is executed with delegatecall on _init
function diamondCut(
Facet[] calldata _diamondCut,
address _init,
bytes calldata _calldata
) external;I am thinking of changing it to this:
interface IDiamondCut {
enum FacetCutAction {Add, Replace, Remove}
struct FacetCut {
address facetAddress;
FacetCutAction action;
bytes4[] functionSelectors;
}
/// @notice Add/replace/remove any number of functions and optionally execute
/// a function with delegatecall
/// @param _diamondCut Contains the facet addresses and function selectors
/// @param _init The address of the contract or facet to execute _calldata
/// @param _calldata A function call, including function selector and arguments
/// _calldata is executed with delegatecall on _init
function diamondCut(
FacetCut[] calldata _diamondCut,
address _init,
bytes calldata _calldata
) external;
event DiamondCut(FacetCut[] _diamondCut, address _init, bytes _calldata);
}
The change is adding replace to the struct.
I have two reasons for possibly making this change:
-
It makes it more explicit what you are doing. I think that contract upgrades should be explicit and intentional. Saying exactly what you are replacing and what you are adding makes it more explicit and intentional.
-
This prevents selector clash. Without this there is a very small possibility of trying to add a function whose signature hashes to the same 4 bytes as an existing function in a diamond, causing an unintentional replacement. With this change, trying to add a selector that already exists would revert.
The downside to making this change is that it slightly complicates the diamondCut function by adding something to it.
I appreciate any feedback on this change, for or against.
The SLOAD on the prior value is done anyway.
Why?
@wjmelements The SLOAD on the prior value is done as a safety check. To make sure that a new function being added does not exist already.
if a diamondCut tries to remove a function that does not exist
That sounds like an error, although benign, to me.
if a diamondCut tries to tries to upgrade a function with a facet it is already using
That sounds benign, to me. Jules
@JulesGoddard You are correct, they are benign.
Would it make more sense to use different functions for the different operations, add, reface, remove? Jules
No. Upgrades need to be atomic or else storage data corruption is possible. For example if new functionality depended on adding 1 function and replacing 1 function then those two changes need to occur in the same transaction. Otherwise it is possible that a transaction is sent inbetween one transaction to add and one transaction to replace.
Embrace zero. No need to make things unnecessarily complicated.
interface IDiamondCut {
struct FacetCut {
bytes4 functionSelector;
address oldProxyTarget; // can be zero for "no proxy"
address newProxyTarget; // can be zero for "no proxy"
}
/// @notice Add/replace/remove any number of functions and optionally execute
/// a function with delegatecall
/// @param _facetCuts Details every change being made
/// @param _initializationContract The address of the contract or facet to execute _initializationCalldata
/// @param _initializationCalldata A function call, including function selector and arguments
/// _calldata is executed with delegatecall on _init
/// @apiSpec Function throws if a specified oldProxyTarget does not match the actual current proxy target
function diamondCut(
FacetCut[] calldata _facetCuts,
address _initializationContract,
bytes calldata _initializationCalldata
) external;
event DiamondCut(FacetCut[] _facetCuts, address _initializationContract, bytes _initializationCalldata);
}
Would the replace flag you propose apply to all the selectors being replaced?
Is that the same as having a separate function to replace selectors?
Yes.
The diamondCut function takes an array of FacetCut structs.
I updated EIP-Diamond Standard with the following:
To add new functions create a
FacetCutstruct withfacetAddressset to the facet that has the new functions andfunctionSelectorsset with the function selectors to add. Set the action enum toAdd.
To replace functions create a
FacetCutstruct withfacetAddressset to the facet that has the replacement functions andfunctionSelectorsset with the function selectors to replace. Set the action enum toReplace.
To remove functions create a
FacetCutstruct withfacetAddressset toaddress(0)andfunctionSelectorsset with the function selectors to remove. Set the action enum toRemove.
That looks great.
Great!
@fulldecent Thank you for your proposed design of FacetCut. Your design is simple and I like it. I appreciate it. But I'm not changing the current design because I think it has similar complexity to the design you proposed and some people have already started using the current design.
Leaving a note on this claiming to solve the issue with maximum contract size. Really any contract that uses an underlying proxy that looks up function signatures for respective implementations is already solving that problem. I spoke about this at the first TruffleCon in 2018 but it is an existing idea used by many and note really owned by anyone in particular.
@elenadimitrova EIP-2535 does not claim to own the idea, it provides a standardized way to use it. I'm sorry if it gives that kind of impression and I am open to changing the text to make it better. It does reference older implementations of the idea in the Inspiration & Development section.
Diamond storage is new and the modularity made possible by it is new, since it wasn't possible until 10 March 2020.
That's great that you spoke about the technique to look up function signatures for respective implementations in 2018. I would like to share that kind of material if possible.
Should diamonds have loupe functions?
I wrote an article about it here: https://dev.to/mudgen/why-loupe-functions-for-diamonds-1kc3
Hello all! I'm here from @trailofbits to let everyone know that we conducted a paid security review of this EIP for a client. Results are forthcoming, however, we felt the need to immediately advise others that they should avoid using this code.
@dguido I'm very interested in this review (and I'm sure others are as well), when can we expect to read about this?
Currently, the words "storage" and "data" are used interchangeably across:
- EIP documentation text
- Variable names in documentation and reference implementation
- Graphics in EIP documentation
For clarity, I request that only one of the terms is used.
On a separate note, I request that there be a base contract class for a "Storage" (or "Data") contract. Benefits include:
- Separation of concerns—the storage may not change after upgrades, so this is beneficial to isolate the storage that will be accessed across old and new facets
- Expressivity—the implementation can better document why storage is using a struct and a assembly (!). As current, an implementation should always prefix every assembly usage with:
This library, although very large, is an implementation of the "Storage" concept in the Diamond Contract standard. You will need to read the entire Diamond specification before reading this large library file in order to properly understand the context of all the functions and data layout here.
Use clear language
Currently, every contract every deployed to Ethereum is a valid EIP-2535: Diamond Standard contract.
This is because the specification does not actually have any requirements.
Please study RFC 2119 and see how this is used in ERC-721 to make a specification that actually specifies things.
Add motivation
Please see EIP-1.
Diamond Standard is complicated. And it is not a valid specification. At the moment this is a fatal flaw. Further, nobody can make sense of it because there is no project scope (i.e. "Motivation").
Please add a motivation section as per EIP-1, so that people can understanding what you are thinking so that contributions can be accepted to fix the things that are broken.
Factor out loupe requirement
The loupe interface may or may not actually be required (Diamond Standard actually does not have any requirements, see above).
But all of the reference implementations support loupe.
The loupe specification (assuming I understand what you mean, not what is written) requires a whole lot of overhead in all diamond contracts which provide no benefit towards the stated goal... I am GUESSING the goal of Diamond Standard is to make it easy to upgrade contracts. (I could make a better argument here if the goal of the project was stated, see MOTIVATION notes above.)
(Weakly) therefore, loupe should be removed or factored out from the main specification because it is complicated and does not support the goal. Also, the reference implementations should include an example ("THE" example) where a loupe and the extra complexity this requires is not in it.
diamontCut not specified
This function is not specified. Multiple implementations are possible which follow the specification (I'm using that word loosely, see above). And the results would be catastrophic.
@fulldecent Thanks for these points. I agree that EIP-2535 Diamond Standard need some informational editing and specifically the motivation section.
The Specification section is meant to be the specification. The diamondCut function is specified in the specification section. In a couple places in the standard the standard says that a diamond must implement the Specification section.
Can you be more specific about what would be catastrophic?
The loupe functions are required. The new Motivation section provides information about why they exists and their use. Also, there is this article about the loupe functions: https://dev.to/mudgen/why-loupe-functions-for-diamonds-1kc3