dev-cloverlab / carpenter

Carpenter is a tool to manage DB schema and data

carpenter

carpenter is a tool to manage DB schema and data inspired by naoina/migu.
By using this, you can manage the database structures and data as text (JSON, CSV).  carpenter can restore the database structure and data from text, or can export them to text in easy.

supported databases are MySQL|MariaDB only currently

Install

% brew tap dev-cloverlab/carpenter
% brew install carpenter

for Gophers.

% go get -u github.com/dev-cloverlab/carpenter

How to use

carpenter has four simple commands are classified database structure and data. For each command is also available to use as indivisually.

Commands for structure

design

design command can export database structure as JSON. By doing below, exports JSON file named table.json to current directory.

% carpenter -s test -d "root:@tcp(127.0.0.1:3306)" design -d ./

When you want to separate files for each tables, you can set -s option.

options:

• -s export JSON files are separated for each table (default off)
• -p pretty output (default off)
• -d export directory path

Each option has alternative long name. Please see the help for details.

build

build command can restore database structure from JSON files. By doing below, generate the difference SQLs between tables and JSON files and execute them.

% carpenter -s test -d "root:@tcp(127.0.0.1:3306)" build -d .

When you want to just show the generated SQLs, you can set --dry-run global option.

Commands for data

export

export command can export data as CSV files. By doing below, export data as CSV files for each table.

% carpenter -s test -d "root:@tcp(127.0.0.1:3306)" export -d .

When you want to select exporting tables, you can set regular expression to -r option like below.

% carpenter -s test -d "root:@tcp(127.0.0.1:3306)" export -r "^master_*$" -d .

import

import command can import CSV files to tables. By doing below, generate the difference SQLs between tables and CSV files and execute them.

% carpenter -s test -d "root:@tcp(127.0.0.1:3306)" import -d .

When you want to just show the generated SQLs, you can set --dry-run global option.

Architecture

Explain how carpenter syncronizes text and database.

MySQL(MariaDB) has information table that has table, column, index and partition information. carpenter refers that and translate it to JSON, and unmarshal it to struct. Both of structs that are made from database and files can compare each field type and etc. When some difference are found for each field, carpenter generates SQLs for resolve differences.

For example:

// about member table

// database
+-------+--------------+------+
| Field | Type         | Null |
+-------+--------------+------+
| name  | varchar(255) | NO   |
| email | varchar(255) | NO   |
+-------+--------------+------+

// file
+--------+--------------+------+
| Field  | Type         | Null |
+--------+--------------+------+
| name   | varchar(255) | NO   |
| email  | varchar(255) | NO   |
| gender | tinyint(4)   | NO   |
+--------+--------------+------+

To generate this.

alter table `member` add `gender` tinyint(4) not null after `email`

carpenter can generate SQLs at various scenes like:

• CREATE
• DROP
• ALTER
• INSERT
• REPLACE
• DELETE

These SQLs are generated by difference of both information structs.

Contribution

1. Fork (https://github.com/dev-cloverlab/carpenter/fork)
2. Create a feature branch
3. Commit your changes
4. Rebase your local changes against the master branch
5. Run test suite with the go test ./... command and confirm that it passes
6. Run gofmt -s
7. Create a new Pull Request

Author

@hatajoe

Licence

MIT