update operation does not take effect
mdzz9527 opened this issue · comments
Please don't check the box below
- I'm aware that bugs and new features should be reported on Apache Jira or Flink mailing list instead of here
Again, please don't check the box below
- I'm aware that new issues on GitHub will be ignored and automatically closed.
请不要勾选以下选项
- 我已知悉缺陷和新功能需要在 Apache Jira 或 Flink 邮件列表中反馈,而不是在这里创建新 issue。
也请不要勾选以下选项
- 我已知悉 GitHub 上的新 issue 会被忽略且自动关闭。
- 环境 flink 1.17.2
- 模式 yarn-session flink SQL
- 使用flink-sql-connector-cdc-oracle-3.1.0flink-connector-jdbc-3.1.2-1.17
- 表都有主键
oracle 的权限配置严格按照 flink cdc 文档配置 https://nightlies.apache.org/flink/flink-cdc-docs-release-3.1/docs/connectors/flink-sources/oracle-cdc/ - SQL 配置如下,可以实现数据全量同步,新增数据同步,删除数据同步,无法同步数据update操作,现象:update 不会删除旧数据,同时会新增一条 ID = 0 的,且只包含变更信息的错误数据;后续测试发现:不管update 多少条数据,也只是不断更新 这条ID=0 的数据
- SQL 如下
`SET
execution.checkpointing.interval = 10min;
create table products (
ID int,
NAME string,
DESCRIPTION string,
WEIGHT float,
primary key (ID) not enforced
) with (
'connector' = 'oracle-cdc',
'hostname' = '10.2.0.79',
'port' = '1521',
'username' = 'flinkuser',
'password' = 'flinkpw',
'database-name' = 'ORCL',
'schema-name' = 'FLINKUSER',
'table-name' = 'PRODUCTS',
'debezium.log.mining.strategy' = 'online_catalog',
'debezium.log.mining.continuous.mine' = 'true',
'connection.pool.size' = '20',
'connect.max-retries' = '3',
'scan.snapshot.fetch.size' = '1024',
'scan.incremental.snapshot.chunk.size' = '8096',
'scan.incremental.snapshot.enabled' = 'true',
'debezium.snapshot.mode' = 'initial'
-- initial
-- schema_only
);
create table tg_products (
ID decimal(9, 0),
NAME string,
DESCRIPTION string,
WEIGHT float,
primary key (ID) not enforced
) with (
'connector' = 'jdbc',
'url' = 'jdbc:mysql://1xxxxxxxxxx:33061/transit_data_oms',
'username' = 'xxxxxxxxx',
'password' = 'xxxxxx',
'table-name' = 'products'
);
insert into
tg_products select * from products;
`
其他依赖
- xdb-19.3.0.0.jar
- ojdbc8-19.3.0.0.jar
- mysql-connector-java-8.0.27.jar
-
oracle version:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for 64-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production -
mysql verison : 5.7.41
flink1.18.1 也稳定复现,
jobmanager_log.txt
这是刚才在flink1.18.1出现这个问题的日志
Thanks @mdzz9527 , could you report the issue in Flink JIRA ?