quixio / quix-streams

Quix Streams - A library for data streaming and Python Stream Processing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Automatic conversion of types improved when adding value to timeseries

peter-quix opened this issue · comments

commented

Tell us about the bug
Python implementation in 0.4.x used to convert types via pythonnet previously in a more automated fashion.
This results in an exception of " Invalid type passed as parameter value." when passing values such as None or numpy.int64.

What did you expect to see?
Investigate what pythonnet did, and how it picked preference for type. Maybe that is something we can do.
Alternatively, None values could be silently ignored if tried to be added. Maybe better ideas in comments?

More details
Previous code:
https://github.com/quixio/quix-streams/blob/v0.4.10/src/PythonClient/lib/quixstreaming/models/parameterdatatimestamp.py#L103
new code:
https://github.com/quixio/quix-streams/blob/v0.5.1/src/PythonClient/src/quixstreams/models/timeseriesdatatimestamp.py#L130

Any workaround?
For the time being the only workaround is to make sure the given value falls into one of str, float, int, bytes, bytearray. So for example None would have to be converted to N/A for float.

commented

The timeseriesdatabuilder is still failing sadly.

Example code:

import numpy

.... some setup code to get a stream producer ...

stream_producer.timeseries.buffer \
            .add_timestamp_nanoseconds(54323000000) \
            .add_value("num_param", numpy.int64(54)) \
            .publish() 

Same for None. Looks like this only got fixed when using non-builder such as converting panda dataframe -> TimeseriesData format