Play Around with Protobuf+LMDB
Try to use protobuf python wrapper to enhance daily working efficiency.
Installation
- option 1. using pre-build
# Ubuntu
sudo apt-get install protobuf-compiler
# Mac
brew install protobuf
After Installation, check the protobuffer compiler
protoc --version
Then install its python library with pip
pip install protobuf
- Option 2. Using source code.. which is not recommanded
git clone https://github.com/google/protobuf.git
# follow its README
Prepare .proto file
ProtoBuffer required a definition file of your own data structure. i.e. the definition.proto file in our example. FYI. this proto file contains a message named Datum, which is copied from CAFFE.
protoc --python_out=. definition.proto
with the 'compile' command, a file will be generated named with **_pb2.py, which will be used in your actual py script
Read & Write
As two scripts shows, reader.py and writer.py
In order to apply the tech into daily working with CAFFE, lmdb support is added.
pip install lmdb
using lmdb in python is pretty easy, please ref to these two py scripts.
with all prerequest, all three steps to play around is as follows.
git clone https://github.com/hyichao/protobuf-lmdb.git
cd protobuf-lmdb
protoc --python_out=. definition.proto
python writer.py
python reader.py