yaobaiwei / MYFS

MYFS: A toy of Distributed File System based on FUSE

This is a README

This is a joint work with Zhao Yunjian.

In case of any problem, welcome to the my repo on github and raise any issue.

There are sample IOZone reports generated using my own computer. The reports are also inside this zip file. One is a text file with named report.txt and the other one is an excel compatible binary file named output.wks.

1. Make sure you have mpi installed in your machines.
   a. Change the C and C++ compiler (CMAKE_C_COMPILER and CMAKE_CXX_COMPILER) in the ./CMakeLists.txt.
   b. For 32 bit machines, change -m64 to -m32 in both CMAKE_CXX_FLAGS and CMAKE_C_FLAGS in the ./CMakeLists.txt.
   
2. Build
   a. Add an environment variable FUSE_ROOT to specify the root directory of fuse.
   b. Add an environment variable MYFS_HOME to specify the directory of this project.
   c. Build the project:
   

   mkdir build
   cd build
   cmake .. -DCMAKE_BUILD_TYPE=Relase
   make all -j4
   

   d. The executable myfs is generated under ../release folder.
3. Change configuration in MYFS_Config.ini
   a. DATANODE_ROOT An available absolute directory on datanode to store the data.
   b. CLIENT_ROOT An auxilary directory on the machine where we mount fuse.
   c. THETA A threshold in bytes to distinguish large file and small file.
   
4. Mount (in case of fire)
   

   cd ../release
   # single machine multi processes
   mpiexec -n 3 ./myfs -s -f [mount_dir]
   # multi machines multi processes
   mpiexec -n 3 -host [hostname_file] ./myfs -s -f [mount_dir]
   

5. Run iozone:

   ./iozone -Rab output.wks -g 2G -i 0 -i 1 -f [mount_dir]/temp
   

6. Unmount
   

   fusermount -u [mount_dir]
   # then remove the `DATANODE_ROOT` on the datanodes.
   

7. Notes:
   a. For Unmount, make sure you remove the DATANODE_ROOT on the datanodes!! Otherwise, you cannot mount again.
   b. Before Mount, make sure you create the [mount_dir] on the machine where we mount fuse!!
   c. For multi mahcines, the first machine is where we mount fuse. Usually we should list local machine as the first machine.