This is a set of small benchmarks for testing quality of code produced by C compilers. In particular the benchmarks are designed to test for specific optimisation capabilities. These are micro-benchmarks and so do not necessarily give an overall view of compiler quality, but it can be interesting to see how different compilers deal with the same input. Therefore, the result times of running the output is arguably less interesting than the actual assembly code. Note that the tests are designed to check for specific optimisations and amplify the performance effect of the tested optimisation; therefore the runtime of one compiler output versus another compiler output can be very different, although in typical real-world programs it's unlikely to ever see such a performance variation for this kind of simple optimisation. This is all in a somewhat rough state. There is little infrastructure, although a contributed bash script to compile and run the benchmarks with different compilers is provided (requires GNU time). Alternatively, you can individually compile and run the benchmarks with the different compilers you are interested in, and time the result. Each directory bm1..bmXX contains one benchmark test. Typically, there is a "driver" program (driver.c) and an additional module which is the benchmark itself (bm1.c, bm2.c, etc). The reason for having two separate files is to prevent whole-program optimization that would skew the results. To get a proper result you should: - compile the driver.c file *once* with any compiler, to produce a "driver.o" object file - compile the benchmark module (bmX.c) with each compiler you want to test and link the result with the driver.o file. eg: gcc -O3 -fomit-frame-pointer -std=c99 driver.o bm1.c - run and time the result in each case: time ./a.out * Just to be clear, to compare two compilers, you should *not* recompile driver.c with the second compiler. The performance of the driver should not be measured. Link-time optimisation should not be used. * Note that the benchmarks are written in the C99 dialect. * Further information about individual benchmarks is usually in the comments in the bmX.c file * If anyone improves the infrastructure or comes up with interesting new test programs, I would appreciate being provided with patches. Davin McCall - davmac@davmac.org