Giters

wavefnx / sigmund

A tool for quickly collecting function selectors and decoding signatures from on-chain EVM bytecode.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

benchmarks results (evmole)

cdump opened this issue · comments

commented

Hey there!
I stumbled upon your project on Twitter and added it to the EVMole benchmarks. It's now in a dedicated branch as WIP: https://github.com/cdump/evmole/tree/sigmund

$ cd benchmarks/
$ PROVIDERS_SELECTORS=sigmund make benchmark-selectors
$ python3 compare.py --providers etherscan sigmund
  mode = selectors
  providers = ['etherscan', 'sigmund']
  datasets = ['largest1k', 'random50k', 'vyper']
  web_listen =
  markdown = False
  show_errors = False

dataset largest1k (1000 contracts, 24427 signatures), sigmund:
  time: 0.5s
  False Positive: 701 signatures, 57 contracts
  False Negative: 120 signatures, 109 contracts

dataset random50k (50000 contracts, 1171102 signatures), sigmund:
  time: 6.8s
  False Positive: 11344 signatures, 1247 contracts
  False Negative: 7550 signatures, 7254 contracts

dataset vyper (780 contracts, 21244 signatures), sigmund:
  time: 0.4s
  False Positive: 57 signatures, 57 contracts
  False Negative: 21244 signatures, 780 contracts

you can compare accurancy and speed (better rerun everything on your hardware) with other selectors extraction tools: https://github.com/cdump/evmole/tree/sigmund?tab=readme-ov-file#function-selectors

commented

Hey!

The project was made mostly for experimentation with different opcode/pattern and it works in a greedy way and especially with the --deep flag. Since it's at very early stage it would make sense to have benchmarks only (and if) a stable version is released in the future.

Thanks for the effort!

commented

very early stage

That's why I put it in dedicated WIP branch and didn't use that results in README.
I thought it would be useful for you, especially at this stage, to understand the accurancy change after implementing new ideas in code.

p.s. I use 'deep=false': https://github.com/cdump/evmole/blob/sigmund/benchmark/providers/sigmund/src/main.rs#L39

commented

Perfect, will check it out when I get some time. Also haven't had time to check your repo but I had someone having an issue in general finding vyper sigs from a contract 0xce6431d21e3fb1036ce9973a3312368ed96f5ce7. Is there any specific flag in evmole for vyper? I can see deep returning some.

Will leave open so you can reply but we can gladly continue on X to keep the issues clean, thanks for the submitting and open-sourcing!