go-xxd
This repository contains my answer to How can I improve the performance of my xxd port? on reddit.
The result is a Go version of xxd that outperforms the native versions on OSX 10.10.1 / Ubuntu 14.04 (inside VirtualBox), see benchmarks below. However, that is not impressive, given that none of the usual xxd flags are supported.
What is interesting however, are the steps to get there:
- Make the code testable and compare against output of native xxd using test/quick: https://github.com/felixge/go-xxd/commit/90262b3dcdc518ca3eaec7171aa14d74d95f34b8
- Fix bugs: https://github.com/felixge/go-xxd/commit/e9ebeb0abdf78f6e7729fdbfc68842b3a86ee0a3, https://github.com/felixge/go-xxd/commit/120804574f12033999f23e6cf6a3b75961f14da1, https://github.com/felixge/go-xxd/commit/dab678ecf5dcb3eff345db8ac68ae6d7438f9d0e
- Buffer output to stdout: https://github.com/felixge/go-xxd/commit/69b5fe0cc7da80d374413d72892507d5e5ecaabc
- Implement a benchmark: https://github.com/felixge/go-xxd/commit/0bce954073ce92b72ed3fbcf36603c6e23852feb
- Use the benchmark + go pprof to get ideas for optimization
- Remove unneeded printf calls: https://github.com/felixge/go-xxd/commit/473330acd320e5318e896d6408fb3d64a5b8e10b
- Optimize hex encoding and avoid allocations: https://github.com/felixge/go-xxd/commit/dce3bca200ae499e1cf57994c7592a42f66694d5, https://github.com/felixge/go-xxd/commit/a48d892de3fb625bdb3e8e367337578c766a42f5, https://github.com/felixge/go-xxd/commit/d653d2f4eeb5fa41e907f260bd95c205bd8e1ff7, https://github.com/felixge/go-xxd/commit/0d3ae7f0be863a138fab3fd2dd89208073b61c7f
- Avoid type casts: https://github.com/felixge/go-xxd/commit/859ebc4489ce81edc0681881700b0f22754943f0
You can also follow along by looking at the commit history: https://github.com/felixge/go-xxd/commits/master
OSX 10.10.1:
xxd native:
$ time xxd image.jpg > /dev/null
real 0m0.205s
user 0m0.202s
sys 0m0.003s
xxd.go (original version from reddit):
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m5.914s
user 0m3.598s
sys 0m2.318s
xxd.go (optimized):
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m0.138s
user 0m0.133s
sys 0m0.004s
Ubuntu 14.04 (inside VirtualBox):
xxd native:
$ time xxd image.jpg > /dev/null
real 0m0.273s
user 0m0.017s
sys 0m0.231s
xxd.go (original version from reddit):
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m5.856s
user 0m3.517s
sys 0m1.897s
xxd.go (optimized):
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m0.233s
user 0m0.021s
sys 0m0.207s