canal-go |中文

Ⅰ.What is canal-go

canal-go is a golang Client of alibaba's open source component Canal,and it provides golang developer a much easier way to use Canal.

Cancal is a incremental Publish&Subscription component based on mysql's binlog.To learn more a bout Cancal,please visit https://github.com/alibaba/canal/wiki

log based incremental pub&sub model can be applied to many business cases:

Database Mirroring
Realtime Database backup
Multi-level index
search build
Busienss cache refresh
Notification on price change(in e-commerce site) and so on

Ⅱ.Using scenarios

canal-go as a golang client of canal,it can be used in situations where Canal is being used.We have listed some using scenarios above,here are some more detailed use cases:

As a substitution of polling Database to monitor Database changes,it is much more effective than polling.
Realtime Update data in search engine when data in mysql database changed.For example,in a E-commerce site,when Product information changed,it can be updated to Elasticsearch、solr,etc in a realtime way.
Realtime update data in cache,Also take E-commerce site as an example,when price,inventory and other product information changed,it can be updated to Redis in a realtime way.
Database Backup,synchronization.
To trigger some other business logics on data change,for example,when a customer place an order without payment in a certain period of time and so the order is cancelled automatically,we get the changed status of the order and then we can push a notification to the customer(or take some other actions)
Push changed data to RabbitMq,kafak or other message queues for consuming.

Ⅲ.Working mechanism

As a golang client of Canal,it use Socket for communication,the transfer Protocol is TCP,and the data exchange Protocol is Google's Protocol Buffer 3.0

Ⅳ.Workflow

Canal connect to Mysql database ,pretented to be a slave of mysql
canal-go(as mentioned it is a client of Cancal) connect to Canal
When data in the database changed,it write the changes to binlog
Canal send dump request to mysql database,get binlog and then parse it.
canal-go send request to Canal for data.
Canal send parsed data to canal-go
canal-go get data successfully and then send an acknowledgement to Canal(optional)
Canal record the consumption position

The following picture shows how it works:

Ⅴ.Quick Start

1. Install Canal

To Install and configure Canal,please visit https://github.com/alibaba/canal/wiki/QuickStart for information

2.Create a golang Console Project

3. Install go get from github source

go get  github.com/CanalSharp/canal-go

or you can install it from nuget manager(a graphic ui) in visual studio

4. Connect to Canal

connector := client.NewSimpleCanalConnector("192.168.199.17", 11111, "", "", "example", 60000, 60*60*1000)
	err :=connector.Connect()
	if err != nil {
		log.Println(err)
		os.Exit(1)
	}
	err = connector.Subscribe(".*\\\\..*")
	if err != nil {
		log.Println(err)
		os.Exit(1)
	}

	for {

		message,err := connector.Get(100, nil, nil)
		if err != nil {
			log.Println(err)
			os.Exit(1)
		}
		batchId := message.Id
		if batchId == -1 || len(message.Entries) <= 0 {
			time.Sleep(300 * time.Millisecond)
			fmt.Println("===没有数据了===")
			continue
		}

		printEntry(message.Entries)

	}

for more detailed information please visit Sample

Ⅵ Run canal-go via Docker quickly

1. Run mysql and canal by command

git clone https://github.com/CanalClient/canal-go.git
cd canal-go
cd docker
docker-compose up -d

2. Use navicat or orther client to connect to mysql

ip：the ip address of the server where docker running

mysql username：root

mysql password：000000

mysql port：4406

There is a default database named test,which contains a table named test

3.run the sample project

4.Test it

Execute the following sql:

insert into test values(1000,'111');
update test set name='222' where id=1000;
delete from test where id=1000;

we can see that after executing insrt,update,delete sql,Our canal-go get the changed data.

Ⅶ What we will do next

canal-go clustering support

Ⅷ Contribute

We gladly accept community contributions.

1.fork the canal-go Project

2.make changes to it

3.make a pull request

Please do not hesitate to make a pull request,your effort will not in vain.

xiaolong321 / canal-go