p-costa / convert-precision

A code to convert the precision of very large binary files from double/single to single/double in parallel, using MPI I/O

Convert Precision

This Fortran program converts large binary files in parallel from/to single to/from double precision, using MPI-I/O. The code assumes that the each file contains only single- or double-precision floating-point numbers.

Usage

1. build the Fortran program:

mpif90 -O3 -D_SINGLE_TO_DOUBLE convert_precision.F90 -o single2double # executable for single-to-double conversion
mpif90 -O3 -D_DOUBLE_TO_SINGLE convert_precision.F90 -o double2single # executable for double-to-single conversion

n.b.: by default, the compiled executable from mpif90 convert_precision.F90 will convert from double to single precision.

2. list the files to be converted in a file files.in. For instance:

ls a.bin b.bin c.bin > files.in

3. run the code, e.g.:

NUM_TASKS=16
mpirun -n $NUM_TASKS ./double2single

4. done! the converted files are generated with the same name as the original file, with the extension .converted appended, i.e., for the example above:

$ ls *.converted
a.bin.converted b.bin.converted c.bin.converted