Large-scale research and development in smart contracts security demand for robust benchmarks. These benchmarks themselves should be based on correct understanding of different vulnerabilities and their detectors. That is exactly what we offer in VulMat. The matrix could be used to design both research and development pipelines in smart contract security tools.
| N | General Category | Detectors and Applications | |||
|---|---|---|---|---|---|
| Target Applications | Slither | Smartcheck | Semgrep | ||
| 1 | Re-entrancy | DeFi | N/A | N/A | compound-borrowfresh-reentrancy |
| Tokenization | N/A | N/A | erc721-reentrancy | ||
| DeFi | N/A | N/A | curve-readonly-reentrancy | ||
| Tokenization | N/A | N/A | erc777-reentrancy | ||
| Tokenization | N/A | N/A | erc677-reentrancy | ||
| N/A | reentrancy-benign | N/A | N/A | ||
| N/A | Reentrancy-events | N/A | N/A | ||
| N/A | reentrancy-no-eth | N/A | N/A | ||
| N/A | reentrancy-unlimited-gas | N/A | N/A | ||
| N/A | reentrancy-eth | N/A | N/A | ||
| 2 | Low-level calls | N/A | N/A | N/A | Arbitrary-low-level-call |
| N/A | N/A | SOLIDITY_CALL_WITHOUT_DATA | N/A | ||
| 3 | Access Control | DeFi | N/A | N/A | Compound-sweeptoken-not-restricted |
| Tokenization | N/A | N/A | Erc20-public-burn | ||
| N/A | N/A | N/A | Accessible-selfdestruct | ||
| DeFi | N/A | N/A | Oracle-price-update-not-restricted | ||
| DeFi | N/A | N/A | Uniswap-callback-not-protected | ||
| Tokenization | arbitrary-send-erc20 | N/A | N/A | ||
| N/A | arbitrary-send-eth | N/A | N/A | ||
| N/A | suicidal | N/A | N/A | ||
| N/A | arbitrary-send-erc20-permit | N/A | N/A | ||
| N/A | N/A | SOLIDITY_TX_ORIGIN | N/A | ||
| 4 | Delegation | N/A | controlled-delegatecall | N/A | Delegatecall-to-arbitrary-address |
| N/A | delegatecall-loop | N/A | N/A | ||
| 5 | Arithmetic | N/A | N/A | N/A | Basic-arithmetic-underflow |
| N/A | divide-before-multiply | SOLIDITY_DIV_MUL | N/A | ||
| 6 | Oracle Manipulation | DeFi | N/A | N/A | Keeper-network-oracle-manipulation |
| 7 | Input Validation | N/A | Missing-zero-check | N/A | N/A |
| 8 | Shadowing | N/A | Shadowing-local | N/A | N/A |
| N/A | shadowing-state | N/A | N/A | ||
| N/A | shadowing-abstract | N/A | N/A | ||
| 9 | Compliance | Tokenization | erc20-interface | N/A | N/A |
| Tokenization | N/A | SOLIDITY_ERC20_TRANSFER_SHOULD_THROW | N/A | ||
| 10 | Timestamp | N/A | timestamp | N/A | N/A |
| N/A | weak-prng | N/A | N/A | ||
| 11 | Initialization | N/A | uninitialized-local | N/A | N/A |
| N/A | uninitialized-state | N/A | N/A | ||
| 12 | Poor Logic Flaws | N/A | Incorrect-equality | SOLIDITY_EXACT_TIME, SOLIDITY_BALANCE_EQUALITY |
N/A |
| N/A | boolean-cst | N/A | N/A | ||
| N/A | N/A | N/A | incorrect-use-of-blockhash | ||
| 13 | Denial of Service | N/A | locked-ether | SOLIDITY_LOCKED_MONEY | N/A |
| N/A | calls-loop | SOLIDITY_TRANSFER_IN_LOOP | N/A | ||
| N/A | msg-value-loop | N/A | N/A | ||
| 14 | State Corruption | N/A | controlled-array-length | SOLIDITY_ARRAY_LENGTH_MANIPULATION | N/A |
| 15 | Function Behavior | N/A | incorrect-modifier | N/A | N/A |
| 16 | External Call Validation | N/A | unchecked-transfer | N/A | N/A |
| N/A | unchecked-lowlevel | N/A | N/A | ||
| 17 | Front-Running | Tokenization | N/A | SOLIDITY_ERC20_APPROVE | N/A |
Any vulnerability directly related to reentrant feature of smart contract execution.
This category is concerned with the way low-level calls are made. Especially, it concerns the way data field of the call is assigned (data source, being checked, etc.).
Detectors that detect any authentication/authorization vulnerability fall into this category. This includes authorizing the function caller and checking their permissions in callee function or setting up an external call with the correct attributes in the caller function.
Delegation is a tricky subject in smart contracts as delegated calls could be exploited by malicious callee to bypass access control mechanism of the contract, and in extreme cases take over the whole contract by setting the state variables pertaining to certain high-permissioned roles.
Any vulnerability and accordingly detector pertaining to the way arithmetic operations are performed and ordered on blockchain.
Exploitations regarding changing the integrity of data gathered with the help of oracles; both in the oracle service contract and the user contract.
Instances of this vulnerability class are failing to perform checks on arbitrary user-supplied data not to be uninitialized or with malicious data in a general sense.
Occurs when a variable from a scope has the same identifier from another scope resulting in the unintended value to be used during runtime.
Failing to comply to the specification of the standard in the contract implementation. The least severe effect will be the interacting contracts/users (who expect the correct complying implementation) will fail to execute external calls/transactions to the function (reverted transactions if external call validation is implemented).
Usage of unreliable/inaccurate blockchain timestamp primitive in critical scenarios such as random number generation or for checking conditions with high accuracy.
Failing to initialize (function-local/state) variables.
This class concerns using conditional statements that are poorly designed, and might not result in intended way. Additionally, relying on poorly designed boolean conditions that could be manipulated by an interacting user leads to an exploit.
Vulnerabilities related to a function in the contract not functioning as intended. Transactions to the vulnerable function might be reverted because of poorly designed usage of loops and msg.* constructs.
The contract state is corrupted or manipulated in unintended ways.
This category encompasses vulnerabilties related to the behavior and flow of execution within functions, including the use of modifiers.
The core issue here is ensuring that transactions or external function calls (especially token transfers) are executed as intended and their outcomes are validated. Proper handling of external calls (transfer, transferFrom, etc.) and checking their return values, which is a key aspect of transaction handling in smart contract development.
Vulnerability to manipulation of final transaction interleaving in blockchain.
- N/A for target application: This means the vulnerability does not pertain to a specific high-level application category.
- Vulnerability detection results that should be used in conjunction: If the vulneraiblity detected by two tools refers to the same bug localization point (same function, same line, same code snippet depending on different detectors) and same type of vulnerability, they should be considered together (for comparison/benchmarking).