lf-edge / ekuiper

Lightweight data stream processing engine for IoT edge

Home Page:https://ekuiper.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to set the indexValue to the latest value?

Forstwith opened this issue · comments

commented

Now I have a source sql stream S1 , and I set the indexValue 1 and indexField id, and I have a rule R1 to log the message from S1.

When I first time to start R1, everything is fine, the R1 logged to indexValue 10 . But when I restart the R1 , the S1 start a new instance, and query start indexValue 1 again, which I want start from indexValue 10.

I read the official documentation, but I didn't find a solution.

The indexValue is a state. It will be persisted if rule qos is set as below example. There are still some limitations, for detail please check https://ekuiper.org/docs/en/latest/guide/rules/state_and_fault_tolerance.html#enable-checkpointing

{
  "id": "ruleReadDB",
  "sql": "SELECT * from sqlDemo",
  "actions": [
    {
      "log": {
      }
    }
  ],
  "options": {
    "qos": 1,
    "checkpointInterval": 60000
  }
}
commented

@ngjaying Thank you very much for your answer, it's exactly what I needed.

But after I set qos=1 and checkPointInterval=60000, when I restart the rule(rule name is 0000002), I get the error.

error: closing rule 0000002 for error: topo 0000002 create store error SQL logic error: near "0000002": syntax error (1)" file="rule/ruleState.go:214

I looked at the source code and it seems that an error occurred when executing the open method.

Here are the logs.

time="2024-02-07T02:44:59Z" level=info msg="Init rule with options &{Debug:false LogFilename: IsEventTime:false LateTol:1000 Concurrency:1 BufferLength:1024 SendMetaToSink:false SendError:true Qos:1 CheckpointInterval:60000 Restart:0x4000ac8b70 Cron: Duration: CronDatetimeRange:[]}" file="planner/planner.go:44"

time="2024-02-07T02:44:59Z" level=info msg="Opening stream" file="topo/topo.go:183" rule=0000002

time="2024-02-07T02:44:59Z" level=error msg="runtime error: topo 0000002 create store error SQL logic error: near "0000002": syntax error (1)" file="infra/saferun.go:56" rule=0000002

time="2024-02-07T02:44:59Z" level=error msg="closing rule 0000002 for error: topo 0000002 create store error SQL logic error: near "0000002": syntax error (1)" file="rule/ruleState.go:214"
`

More info:

  1. ekuiper version: 1.12.3-alpine
  2. Run using docker container.

Could you try another rule id like rule000002?

commented

@ngjaying After change rule id to rule000002, works fine. Thank you very much.

Do you think this is a bug? Because the 0000002 rule id will cause failure to start rule when the Qos setting is not 0

Kind of, but we will recommend to not set id to start with number. We are adding rule id validation.

commented

Okay, then this issue can be closed.
Again, thank you very much for your help.