NewStore / mercator

Python DSL to leverage translation of dictionaries and SQLAlchemy into Protobuf objects.

Mercator - Data Mapper for Protobuf

version 0.1.11

Python DSL to leverage translation of dictionaries and SQLAlchemy into Protobuf objects

What is Mercator ?

Mercator is a Python library that simplifies the following of serializing dictionary data into Protobuf binary data.

Mercator actually supports extracting data from:

• dictionaries
• SQLAlchemy model instances
• Any opaque python objects (e.g.: collections.namedtuple)

When should I use Mercator ?

• When migrating custom implementations of RPC to gGRPC.
• When migrating in-memory data to Protobuf.

When should I not use Mercator ?

• When writing gRPC services from scratch.
• When writing anything that uses Protobuf gets called.

Install

pip install mercator

Documentation

https://mercator.readthedocs.org

Basic Usage

1. Given a protobuf declaration

syntax = "proto3";
package services.social_platform;

import "google/protobuf/timestamp.proto";

message User {
  message AuthToken {
    string value = 1;
    google.protobuf.Timestamp created_at = 2;
    google.protobuf.Timestamp expires_at = 3;
  }
}

2. Declare Mappings

from mercator import (
    ProtoMapping,
    ProtoKey,
    ProtoList,
    SinglePropertyMapping,
)
from google.protobuf.timestamp_pb2 import Timestamp
from google.protobuf.struct_pb2 import Struct
from google.protobuf import struct_pb2


ProtobufTimestamp = SinglePropertyMapping(int, Timestamp, 'seconds')

class UserAuthTokenMapping(ProtoMapping):
    __proto__ = domain_pb2.User.AuthToken
    value = ProtoKey('data', str)
    created_at = ProtoKey('created_at', ProtobufTimestamp)
    expires_at = ProtoKey('expires_at', ProtobufTimestamp)


class UserMapping(ProtoMapping):
    __proto__ = domain_pb2.User

    tokens = ProtoList('tokens', UserAuthTokenMapping)

3. Generate python files

python -m grpc_tools.protoc -I ./ --python_out=./ --grpc_python_out=./ ./*.proto

4. Process data!

info = {
    'login': 'Hulk',
    'email': '[e-mail]',
    'tokens': [
        {
            'data': 'this is the token',
            'created_at': 1552240433,
            'expires_at': 1552240733,
        }
    ],
}

user = UserMapping(info).to_protobuf()

assert isinstance(user, domain_pb2.User)

Contributing

1. Check the code structure documentation
2. Write tests
3. Write code
4. Send a pull-request