siddhi-io / siddhi

Stream Processing and Complex Event Processing Engine

Home Page:http://siddhi.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

siddhi cann't cache redis data

puroc opened this issue · comments

commented

Description:

siddhi cann't cache redis data,it will get data from redis when it process every event

Affected Siddhi Version:
5.1.2

OS, DB, other environment details and versions:
OS:mac
DB:redis

Steps to reproduce:

`
@app:name("one-two")

@source(type='kafka',
topic.list='one-two',
partition.no.list='0',
threading.option='single.thread',
group.id='group',
is.binary.message='true',
bootstrap.servers='10.10.30.200:9092',
@Map(type='json',@attributes(cmCode = "cmCode", orgId = "orgId",
timestamp = "timestamp",temperature="data.temperature")))
define stream DataStream(cmCode string,orgId string,timestamp long,temperature int);

@store(type='redis',host='localhost', table.name="SourceDeviceTable12",port='6379',@cache(size="100", cache.policy="LRU",retention.period="5 min", purge.interval="1 min"))
@PrimaryKey('cmCode')
define table SourceDeviceTable12(cmCode string,addressId string);

@store(type='redis',host='localhost', table.name="DestDeviceTable12",port='6379')
@PrimaryKey('cmCode')
define table DestDeviceTable12(cmCode string,addressId string,temperatureBeforeTenMin int);

@sink(type='log')
define stream LowProductionAlertStream(cmCode string,orgId string,timestamp long,temperature int);

@info(name='insertQuery1')
from DataStream[SourceDeviceTable12.cmCode == cmCode in SourceDeviceTable12]
select * insert into SourceDataStream;

@info(name='insertQuery2')
from SourceDataStream as t1 join DestDeviceTable12 as t2
select t1.cmCode,t1.orgId,t1.timestamp,t1.temperature having (t1.cmCode==t2.cmCode and t1.temperature >= t2.temperatureBeforeTenMin+5)
insert into LowProductionAlertStream;

`
Related Issues:

commented

when i run this siddhi app,i saw the redis log as below,i found siddhi get data for every event and the cache is not useful

1631016278.191435 [0 127.0.0.1:56335] "SCAN" "0" "match" "DestDeviceTable12:" "count" "1000"
1631016278.191570 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:4"
1631016278.191654 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:4"
1631016278.191751 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:5"
1631016278.191831 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:5"
1631016283.199265 [0 127.0.0.1:56336] "HGETALL" "SourceDeviceTable12:4"
1631016283.199527 [0 127.0.0.1:56335] "SCAN" "0" "match" "DestDeviceTable12:
" "count" "1000"
1631016283.199648 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:4"
1631016283.199787 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:4"
1631016283.199875 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:5"
1631016283.199960 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:5"
1631016288.211587 [0 127.0.0.1:56336] "HGETALL" "SourceDeviceTable12:5"
1631016288.211866 [0 127.0.0.1:56335] "SCAN" "0" "match" "DestDeviceTable12:" "count" "1000"
1631016288.212013 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:4"
1631016288.212152 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:4"
1631016288.212277 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:5"
1631016288.212367 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:5"
1631016293.218661 [0 127.0.0.1:56336] "HGETALL" "SourceDeviceTable12:4"
1631016293.218882 [0 127.0.0.1:56335] "SCAN" "0" "match" "DestDeviceTable12:
" "count" "1000"
1631016293.219011 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:4"
1631016293.219131 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:4"
1631016293.219248 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:5"
1631016293.219345 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:5"
1631016298.227721 [0 127.0.0.1:56336] "HGETALL" "SourceDeviceTable12:5"
1631016298.228214 [0 127.0.0.1:56335] "SCAN" "0" "match" "DestDeviceTable12:*" "count" "1000"
1631016298.228565 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:4"
1631016298.228757 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:4"
1631016298.228897 [0 127.0.0.1:56335] "TYPE" "DestDeviceTable12:5"
1631016298.229077 [0 127.0.0.1:56335] "HGETALL" "DestDeviceTable12:5"

Currently, the caching is only supported for the store types supported from the siddhi-store-rdbms extension