samhuangszu / godfs

A simple fast, easy use distributed file system written by golang.

godfs

README | 中文文档

Description

godfs is a simple fast, easy use distributed file system written by golang.

godfs provides out-of-the-box usage and friendly support for docker，

You can pull the image on docker hub: https://hub.docker.com/r/hehety/godfs/

Features

• Fast, lightweight, stable, out-of-the-box, friendly api.
• Easy to expand, Stable to RUN.
• Low resource overhead.
• Native client api and java client api(not started).
• API for http upload and download.
• Clear logs help troubleshoot errors.
• Support different platforms: Linux, Windows, Mac
• Better support docker.
• File fragmentation storage.
• Perfect data migration solution.
• Support readonly node.
• File synchronization in same group.

Install

Please install golang1.9+ first!

Take CentOS 7 as example.

build from latest source code:

yum install golang -y
git clone https://github.com/hetianyi/godfs.git
cd godfs
./make.sh

After the build is successful, three files will be generated under the ./bin directory:

./bin/client
./bin/storage
./bin/tracker

Install godfs binary files to /usr/local/godfs:

./install.sh /usr/local/godfs

You can start tracker server by:

/usr/local/godfs/bin/tracker [-c /your/tracker/config/path]

and start storage node by:

/usr/local/godfs/bin/storage [-c /your/storage/config/path]

then you can using command client directly in command line to upload and download file.

For example:

# upload a file
client -u /you/upload/file

You can upload file by:

 /e/godfs-storage/client/bin/client -u /f/project.rar

also, it's cool that you can upload all files in a directory by:

echo \"$(ls -m /f/foo)\" |xargs client -u

if you don't has a godfs client, you can use curl to upload files by:

curl -F "file=@/your/file" "http://your.host:81/upload"

# download a file as 123.zip
client -d G01/10/M/2c9da7ea280c020db7f4879f8180dfd6 -n 123.zip

build docker image from latest source code:

cd godfs/docker
docker build -t godfs .

It is highly recommended to use docker to run godfs. You can pull the docker image from docker hub:

docker pull hehety/godfs

start tracker using docker:

docker run -d -p 1022:1022 --name tracker --restart always -v /godfs/data:/godfs/data --privileged -e log_level="info" hehety/godfs:latest tracker

start storage using docker:

docker run -d -p 1024:1024 -p 80:8001 --name storage -v /godfs/data:/godfs/data --privileged -e trackers=192.168.1.172:1022 -e bind_address=192.168.1.187 -e port=1024  -e instance_id="01" hehety/godfs storage
# you'd better add docker command '-e port=1024' on single machine.  

we're here using directory /godfs/data to persist data.

client usage:

-u string 
    the file to be upload, if you want upload many file once, quote file paths using """ and split with ","
    example:
    client -u "/home/foo/bar1.tar.gz, /home/foo/bar1.tar.gz"
-d string 
    the file to be download
-l string 
    custom logging level: trace, debug, info, warning, error, and fatal
-n string 
    custom download file name
--set string
    set client config, for example: 
    client --set "tracker=127.0.0.1:1022"
    client --set "log_level=info"