apache / flink-cdc

Flink CDC is a streaming data integration tool

Home Page:https://nightlies.apache.org/flink/flink-cdc-docs-stable

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

update operation does not take effect

mdzz9527 opened this issue · comments

commented

Please don't check the box below

  • I'm aware that bugs and new features should be reported on Apache Jira or Flink mailing list instead of here

Again, please don't check the box below

  • I'm aware that new issues on GitHub will be ignored and automatically closed.

请不要勾选以下选项

  • 我已知悉缺陷和新功能需要在 Apache Jira 或 Flink 邮件列表中反馈,而不是在这里创建新 issue。

也请不要勾选以下选项

  • 我已知悉 GitHub 上的新 issue 会被忽略且自动关闭。
commented
  1. 环境 flink 1.17.2
  2. 模式 yarn-session flink SQL
  3. 使用flink-sql-connector-cdc-oracle-3.1.0flink-connector-jdbc-3.1.2-1.17
  4. 表都有主键
    oracle 的权限配置严格按照 flink cdc 文档配置 https://nightlies.apache.org/flink/flink-cdc-docs-release-3.1/docs/connectors/flink-sources/oracle-cdc/
  5. SQL 配置如下,可以实现数据全量同步,新增数据同步,删除数据同步,无法同步数据update操作,现象:update 不会删除旧数据,同时会新增一条 ID = 0 的,且只包含变更信息的错误数据;后续测试发现:不管update 多少条数据,也只是不断更新 这条ID=0 的数据
  6. SQL 如下
    `SET
    execution.checkpointing.interval = 10min;
    create table products (
    ID int,
    NAME string,
    DESCRIPTION string,
    WEIGHT float,
    primary key (ID) not enforced
    ) with (
    'connector' = 'oracle-cdc',
    'hostname' = '10.2.0.79',
    'port' = '1521',
    'username' = 'flinkuser',
    'password' = 'flinkpw',
    'database-name' = 'ORCL',
    'schema-name' = 'FLINKUSER',
    'table-name' = 'PRODUCTS',
    'debezium.log.mining.strategy' = 'online_catalog',
    'debezium.log.mining.continuous.mine' = 'true',
    'connection.pool.size' = '20',
    'connect.max-retries' = '3',
    'scan.snapshot.fetch.size' = '1024',
    'scan.incremental.snapshot.chunk.size' = '8096',
    'scan.incremental.snapshot.enabled' = 'true',
    'debezium.snapshot.mode' = 'initial'
    -- initial
    -- schema_only
    );

create table tg_products (
ID decimal(9, 0),
NAME string,
DESCRIPTION string,
WEIGHT float,
primary key (ID) not enforced
) with (
'connector' = 'jdbc',
'url' = 'jdbc:mysql://1xxxxxxxxxx:33061/transit_data_oms',
'username' = 'xxxxxxxxx',
'password' = 'xxxxxx',
'table-name' = 'products'
);

insert into
tg_products select * from products;
`

commented

其他依赖

  1. xdb-19.3.0.0.jar
  2. ojdbc8-19.3.0.0.jar
  3. mysql-connector-java-8.0.27.jar
commented
  1. oracle version:
    Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    PL/SQL Release 11.2.0.1.0 - Production
    CORE 11.2.0.1.0 Production
    TNS for 64-bit Windows: Version 11.2.0.1.0 - Production
    NLSRTL Version 11.2.0.1.0 - Production

  2. mysql verison : 5.7.41

commented

flink1.18.1 也稳定复现,
jobmanager_log.txt
这是刚才在flink1.18.1出现这个问题的日志

Thanks @mdzz9527 , could you report the issue in Flink JIRA