Auditing Department: per contract reward calculation
Dexaran opened this issue · comments
Abstract
The following describes the proposed payment scheme for security auditors for their work. The developed scheme is universal and is based on the length of the code of the auditable smart contract.
Motivation
One of the main goals of Callisto is to develop a self-sustaining DAO-like security auditing department structure. Therefore, the payment schema should ensure a high level of automatization and formalization.
The previously used reward calculation formula relied on "contract complexity" variable, which was manually assigned by security auditing manager. As we plan to scale the structure of security department, we need to introduce a flexible way of contract complexity evaluation.
Therefore, it is proposed to use the length of the contract code as a measure of the complexity of the contract.
Specification
In the Security Department of Callisto, smart-contract auditors are paid once a month, on the 15th day of each month.
The total amount of payment is calculated based on the amount of tasks performed during the last month. Each security audit is evaluated separately and a security auditor receives payment for each audit performed.
Each finding has a certain weight in points. The following values will be used to evaluate findings according to its severity:
| Severity | Weight in points |
|---|---|
| Critical | 40 |
| High | 16 |
| Medium | 4 |
| Low | 1 |
The following formula is used to calculate the auditor's reward for the assigned task:
Where:
reward - the amount of CLO that will be paid to the auditor for his(her) contribution to this security audit.
audit reward = 50 * [number of lines]
sum (auditor points) - all points earned by the auditor.
sum (total points) - maximum points for all issues reported by all auditors who worked on this security audit.
The [number of lines] of code in the source code of the auditable smart-contract which is calculated excluding empty lines and comments. SLOC Counter will be used for this purpose.
Auditors will receive the reward depending on the quality and quantity of the work done.
If a contract has only low severity issues or no issues then it’s reward will be divide equally between all auditors who worked at the security audit of this contract.
I think that the new proposed formula is the most suitable of all the proposed formulas, however the parameters values should be discussed more.
Previously I proposed the minimal threshold and the previous version of the formula looked like this:
reward = 10 000 + 20 000 * floor( (number_of_lines / 400) )
The presence of the minimum threshold is based on the assumption that there is always a certain level of work that should be done, i.e. understanding the purpose of the contract and what this program should do.
This formula was rejected becuase the most common contracts are ICOs and tokens, which are standardized and require almost no preliminary research to proceed with the audit.
-
How are the
auditor pointscalculated? I think it's worth revealing in this proposal. -
Often the complexity increases not in arithmetic progression. Much easier to make 4 audits with a 250 rows than 1 with a 1000 rows. For high-quality verification of contracts more than 600-700 lines need to spend considerable time. In such contracts, there are much more states and relationships between the elements.
I am agree with Alexander about item 2.
How will calculate the value of sum (total points) including minor observation, low and medium issues? Their weight in this formula.
@gorbunovperm @danbogd you're right, I've added the information about point weights.
I will talk and about myself only, if any other auditor agree with me he can let us know.
We have previously agreed about being paid in USD or CLO, I have selected CLO for many reasons and one of them is because I believe that CLO price will raise sooner or later.
The new formula will reduce the reward which is indirectly breaking the aggreement that we had. since we have been paid in average 2.5 times less clo for the same amount of work. in my opinion the amount of the reward should stay the same as now but using the new formula.
I agree with the new formula auditor earnings will be less. I would be satisfied with the payment which is valid now. You can enter an additional gradation by low level errors, as an added bonus.