jercheng / zpd

A simple SQL-like database built on top of Key-Value storages

ZPD

• ZPD
  • Overview
  • Architecture
  • Requirement
  • Install
  • Build
  • Run
  • Test
  • Document
  • Contribution
  • Acknowledgments
  • License

Overview

ZPD (or ZaloPay Database) is a probationary challenge performed at ZaloPay. It's an experiment to gain deep knowledge about handling simple SQL queries and how to glue them with a key-value storage.

ZPD is implemented using Golang as a gRPC service and built on top of following frameworks:

• Consul for leader elections.

• TiKV as the main key-value storage

Architecture

The architecture of the ZPD:

Including layers:

• Connection layer
• Parser layer
• Core layer:
  • Executor
  • Consul Agent
  • Bridge API
• Data access layer
• Storage:
  • TiKV client layer

See the architecture of ZPD here.

Requirement

• Golang version >= 1.12
• gRPC
• Docker version >= 17.06 and Docker Compose
• Locust

Install

• Install Golang
• Install gRPC
• Install Docker
• Install Locust
• Install TiKV and PD using Docker Compose
• Install Consul bằng Docker.

Build

Clone this project:

# Clone
$ git clone https://gitlab.zalopay.vn/zpx-core-team/tidb-internals.git

Run

ZPD, PD, TiKV, Consul are all built with Docker compose. Just go to the docker-compose folder and run docker-compose up.

# go to the folder docker-compose
$ cd ./tidb-layer/source/docker-compose

# Run docker-compose
$ docker-compose up

Test

Must run ZPD as the section above, then run tests of the APIs:

# go to cmd/client
$ cd ./tidb-layer/source/cmd/client

# run tests
$ go test -run TestClientExecuteCreateDB 

• You can write more tests into client_test.go like the available format.

Document

Read more document about ZPD:

• Specific description
• Sequence diagram
• Architecture ZPD
• Overview ZPD

Read blogs:

• SQL Planning: Parser & Optimizer
• SWIM: Protocol of friends
• Building a SQL Database using key-value storage

Contribution

This project was built by AJPham and Alex Nguyen under the guidance from Anh Le (Andy).

Acknowledgments

• This project used open source:
  • xwb1989/sqlparser: a SQL parser.
  • Consul: handle leader election.
  • PingCap: support PD and TiKV parts.

License

See the LICENSE file for license rights and limitations.