Database Changes Finder
DatabaseChangesFinder is an application that measures changes from a specific point in time based on statistical information of a database and obtains differential data. The information is predictive from statistics and table column naming conventions. Please use it as a reference value.
It is specific to PostgreSQL statistics and is not available for other databases.
Setup
environment variable
Set the connection information for the database used by Database Changes Finder in an environment variable. You should be able to connect from within the docker environment.
$ export DATABASE_URL=pgsql://user:pass@host_name:5432/database_name
docker build
$ docker-compose build
$ docker-compose up -d
composer install
$ docker-compose run phpcli74 composer install
Usage
make snapshot data
Save preoperational database statistics to a file
php DatabaseChangesFinder.php start transactionKey
The transactionKey is a string used to identify the start and end pairs. As it is used as part of the file name, it will not work if you try to use characters that are not available in the file name. The file dcf.transactionKey.json is created after the command is executed.
get changes from snapshot
Acquire the difference of statistical information before and after the operation, and estimate and extract the updated data in the database based on the difference.
php DatabaseChangesFinder.php end transactionKey
example
$ docker-compose run phpcli74 php DatabaseChangesFinder.php start 202010111200
:
File dcf.202010111200.json is created.
:
some kind of operations
ex. exec login action
:
$ docker-compose run phpcli74 php DatabaseChangesFinder.php end 202010111200
{
"accounts": {
"update_count": 1,
"data": [
{
"id": 193,
"family_name": "Uno",
"first_name": "Kazuhiko",
"last_login": "2020-10-11 21:56:15",
"created": "2020-10-10 08:50:10",
"creator": 1,
"updated": "2020-10-11 21:56:15",
"updater": 193,
"deleted": null,
"deleter": null,
}
]
},
"login_logs": {
"insert_count": 1,
"data": [
{
"id": 100,
"account_id": 193,
"created": "2020-10-11 21:56:15",
"creator": 193,
}
]
}
}
TODO
- Refactoring code
- Add unit tests.