"enclose.by.quotes" not functioning
Hypnotec opened this issue · comments
E> It appears that 'enclose.by.quotes' does not function when using 'custom.query'
E> I might advocate that anyone who comes across this project consider switching to StreamSets, as Flume doesn't support RDBMS and StreamSets provides support for a large array of sources and targets without relying on third-party support.
Does the following property, "enclose.by.quotes", not function when using PSQL? I am not able to get it to work while using the following Flume properties:
#
# SOURCE
agent.sources.source-sql.type = org.keedio.flume.source.SQLSource
agent.sources.source-sql.channels = channel-file
agent.sources.source-sql.status.file.path = /home/admin/.tmp/.flume
agent.sources.source-sql.status.file.name = source-sql.messages.status
agent.sources.source-sql.hibernate.connection.url = jdbc:postgresql://localhost:5432/test
agent.sources.source-sql.hibernate.connection.user = postgres
agent.sources.source-sql.hibernate.connection.password = ''
agent.sources.source-sql.hibernate.connection.driver_class = org.postgresql.Driver
# NOTE: 'message_time' is integer, 'message' is text
agent.sources.source-sql.custom.query = select message_time, message from messages where message_time > $@$ order by message_time
agent.sources.source-sql.enclose.by.quotes = false
agent.sources.source-sql.run.query.delay = 5000
agent.sources.source-sql.batch.size = 500
agent.sources.source-sql.max.rows = 10000
Currently, my work around is to remove all instances of quotes through an interceptor, but I am not a huge fan of this approach as users may input double-quotes - which the 'message' field would capture.
Hi Hypnotec,
when trying to reproduce like issue #44, we get values with double quotes.
- can you please , version of flume-ng-sql
- as i see in you config, you are using default delimiter "," can you try a different one?
best, Luis