support protobuf c++ implementation?
yuz11 opened this issue · comments
I pull a large number of data(70w+ rows),found it pretty slow when it runs to
client.py:506 (1~2 second dealing with one bugket data in a fetch())
response_data = self._apply(request)
response = responses_pb2.FetchResponse()
response.ParseFromString(response_data)
I guess pure python protobuf make it slow.
python protobuf c++ implement may help
would u share the .proto files so that I can make a test by myself?
Regards,
Rafer
You can get the PB files here:
https://github.com/apache/calcite-avatica/tree/master/core/src/main/protobuf
thx~I'll take a shot
u can close this issue now. I just run a test, protobuf with c++ implement is about 10 times faster than pure python version.But it's just a matter of python protobuf lib version, do not have to rebuild the message.
python 3.0 or ++ pip install will do; python 2.x, will have to setup by source code.
Regards.Sorry for taking this long.Have been busy these days