Candle Data with arctic db
BlackWingedKing opened this issue · comments
How do I get candle data from BINANCE? (I'm saving the data in arctic)
I have followed the config given in this PR
here is my config.yaml exchanges field
BINANCE:
retries: -1
channel_timeouts:
candles: 30
l2_book:
symbols: [BTC-USDT]
book_delta: true
book_interval: 10000
trades: [BTC-USDT]
ticker: [BTC-USDT]
candles: [BTC-USDT]
The error which I get
2021-05-14 07:32:43,596 : INFO : Reading BINANCE-l2_book-BTC-USDT
2021-05-14 07:32:43,602 : INFO : l2_book-BINANCE-BTC-USDT: Read 580 messages from Redis
2021-05-14 07:32:43,708 : INFO : l2_book-BINANCE-BTC-USDT: Removed 580 entries through id 1620957763528-0
2021-05-14 07:32:43,708 : INFO : Write Complete BINANCE-l2_book-BTC-USDT
2021-05-14 07:32:43,708 : INFO : Reading BINANCE-trades-BTC-USDT
2021-05-14 07:32:43,723 : INFO : trades-BINANCE-BTC-USDT: Read 1410 messages from Redis
2021-05-14 07:32:43,770 : INFO : trades-BINANCE-BTC-USDT: Removed 1410 entries through id 1620957763615-0
2021-05-14 07:32:43,770 : INFO : Write Complete BINANCE-trades-BTC-USDT
2021-05-14 07:32:43,771 : INFO : Reading BINANCE-ticker-BTC-USDT
2021-05-14 07:32:43,793 : INFO : ticker-BINANCE-BTC-USDT: Read 3784 messages from Redis
2021-05-14 07:32:43,886 : INFO : ticker-BINANCE-BTC-USDT: Removed 3784 entries through id 1620957763670-0
2021-05-14 07:32:43,887 : INFO : Write Complete BINANCE-ticker-BTC-USDT
2021-05-14 07:32:43,887 : INFO : Reading BINANCE-candles-BTC-USDT
2021-05-14 07:32:43,887 : INFO : candles-BINANCE-BTC-USDT: Read 29 messages from Redis
2021-05-14 07:32:43,902 : ERROR : Aggregator running on PID 16811 died due to exception
Traceback (most recent call last):
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/aggregator/aggregator.py", line 108, in loop
store.write(exchange, dtype, pair, time.time())
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/data/storage.py", line 37, in write
s.write(exchange, data_type, pair, timestamp)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/data/arctic.py", line 63, in write
self.con[exchange].append(f"{data_type}-{pair}", df, upsert=True, chunk_size=chunk_size)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/arctic/chunkstore/chunkstore.py", line 517, in append
return self.write(symbol, item, metadata=metadata, audit=audit, **kwargs)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/arctic/chunkstore/chunkstore.py", line 355, in write
for start, end, chunk_size, record in chunker.to_chunks(item, **kwargs):
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/arctic/chunkstore/date_chunker.py", line 45, in to_chunks
period_obj = dates.to_period(chunk_size)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/pandas/core/accessor.py", line 99, in f
return self._delegate_method(name, *args, **kwargs)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/pandas/core/indexes/datetimelike.py", line 905, in _delegate_method
result = operator.methodcaller(name, *args, **kwargs)(self._data)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py", line 1112, in to_period
"You must pass a freq argument as current index has none."
ValueError: You must pass a freq argument as current index has none.
Task exception was never retrieved
future: <Task finished coro=<Aggregator.loop() done, defined at /home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/aggregator/aggregator.py:43> exception=ValueError('You must pass a freq argument as current index has none.')>
Traceback (most recent call last):
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/aggregator/aggregator.py", line 108, in loop
store.write(exchange, dtype, pair, time.time())
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/data/storage.py", line 37, in write
s.write(exchange, data_type, pair, timestamp)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/cryptostore/data/arctic.py", line 63, in write
self.con[exchange].append(f"{data_type}-{pair}", df, upsert=True, chunk_size=chunk_size)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/arctic/chunkstore/chunkstore.py", line 517, in append
return self.write(symbol, item, metadata=metadata, audit=audit, **kwargs)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/arctic/chunkstore/chunkstore.py", line 355, in write
for start, end, chunk_size, record in chunker.to_chunks(item, **kwargs):
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/arctic/chunkstore/date_chunker.py", line 45, in to_chunks
period_obj = dates.to_period(chunk_size)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/pandas/core/accessor.py", line 99, in f
return self._delegate_method(name, *args, **kwargs)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/pandas/core/indexes/datetimelike.py", line 905, in _delegate_method
result = operator.methodcaller(name, *args, **kwargs)(self._data)
File "/home/ritesh/.conda/envs/cryptostore/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py", line 1112, in to_period
"You must pass a freq argument as current index has none."
ValueError: You must pass a freq argument as current index has none.
@bmoscon can you look into this error?
Hi @BlackWingedKing, does this work for you, managed to solve it ?