Flume restarts with custom query pulls data from beginnning only

[Aravind Krishnan]

Hi ,

I am using Cloudera distribution and my Flume version is 1.6 as mentioned in the CDH

flume-ng-1.6.0+cdh5.11.1+160

Custom query to an Oracle sql source is not working as expected. Below is the configuration for custom query
agent.sources.sql-source.type = org.keedio.flume.source.SQLSource
agent.sources.sql-source.start.from = 62010
agent.sources.sql-source.custom.query = select id , columns from (select id,columns,rownum as iden fromsometable) where iden > $@$

agent.sources.sql-source.status.file.path =
agent.sources.sql-source.status.file.name = sql-source-fail-recovery.status
agent.sources.sql-source.status.column = ID

My ID is not a primary key and hence I am using rownum from oracle.

My status file does not have any details it just has a {}. After each restart , I am seeing that flume pulls record from the start.

[Luis Lázaro]

Hi akrishnankogentix,
can you please?

• flume-ng-sql version.
• log from flume service from Cloudera.

best