Giters

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/

Home Page:https://pingcap.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Adding an index failed with DXF enabled and max-index-length set in one TiDB node.

King-Dylan opened this issue · comments

commented

Bug Report

Adding an index failed with DXF enabled and max-index-length set in one TiDB node.

1. Minimal reproduce step (Required)

1. enable DXF
2. set max-index-length = 10000 in one of tidb nodes randomly, and create index.
CREATE TABLE `t1` (
  `item_primary_key` varbinary(255) NOT NULL,
  `item_id` varchar(45) DEFAULT NULL,
  `normalized_short_link_clusters` json DEFAULT NULL,
  `point_of_sale_country` varchar(2) DEFAULT NULL,
  PRIMARY KEY (`item_primary_key`) /*T![clustered_index] CLUSTERED */,
);
CREATE INDEX idx_test on t1 ((cast(normalized_short_link_clusters as char(8000) array)),point_of_sale_country);

2. What did you expect to see? (Required)

No panic logs were printed in the DDL owner's log, and no cancellation information for DDL was printed in the command terminal.

3. What did you see instead (Required)

mysql> CREATE INDEX idx_test on t1 ((cast(normalized_short_link_clusters as char(8000) array)),point_of_sale_country);
ERROR 8214 (HY000): Cancelled DDL job
ddl owner:
[2024/05/15 09:36:11.581 +08:00] [ERROR] [misc.go:113] ["panic in the recoverable goroutine"] [label=ddl-worker] [funcInfo="worker 17, tp add index runDDLJob"] [r="runtime error: invalid memory address or nil pointer dereference"] [stack="github.com/pingcap/tidb/pkg/util.Recover\n\t/workspace/source/tidb/pkg/util/misc.go:117\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:914\nruntime.panicmem\n\t/usr/local/go/src/runtime/panic.go:261\nruntime.sigpanic\n\t/usr/local/go/src/runtime/signal_unix.go:861\ngithub.com/pingcap/tidb/pkg/ddl.suppressErrorTooLongKeyKey\n\t/workspace/source/tidb/pkg/ddl/index.go:86\ngithub.com/pingcap/tidb/pkg/ddl.buildIndexColumns\n\t/workspace/source/tidb/pkg/ddl/index.go:131\ngithub.com/pingcap/tidb/pkg/ddl.BuildIndexInfo\n\t/workspace/source/tidb/pkg/ddl/index.go:307\ngithub.com/pingcap/tidb/pkg/ddl.(*worker).onCreateIndex\n\t/workspace/source/tidb/pkg/ddl/index.go:613\ngithub.com/pingcap/tidb/pkg/ddl.(*worker).runDDLJob\n\t/workspace/source/tidb/pkg/ddl/ddl_worker.go:1240\ngithub.com/pingcap/tidb/pkg/ddl.(*worker).HandleDDLJobTable\n\t/workspace/source/tidb/pkg/ddl/ddl_worker.go:930\ngithub.com/pingcap/tidb/pkg/ddl.(*ddl).delivery2Worker.func1\n\t/workspace/source/tidb/pkg/ddl/job_table.go:462\ngithub.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).Run.func1\n\t/workspace/source/tidb/pkg/util/wait_group_wrapper.go:157"]
[2024/05/15 09:36:11.600 +08:00] [INFO] [ddl_worker.go:1169] ["run DDL job"] [worker="worker 18, tp add index"] [category=ddl] [jobID=106] [conn=136314882] [category=ddl] [job="ID:106, Type:add index, State:cancelling, SchemaState:none, SchemaID:2, TableID:104, RowCount:0, ArgLen:0, start time: 2024-05-15 09:36:11.531 +0800 CST, Err:<nil>, ErrCount:1, SnapshotVersion:0, LocalMode: false, UniqueWarnings:0"]
[2024/05/15 09:36:11.604 +08:00] [INFO] [rollingback.go:547] ["the DDL job is cancelled normally"] [worker="worker 18, tp add index"] [category=ddl] [jobID=106] [conn=136314882] [job="ID:106, Type:add index, State:cancelled, SchemaState:none, SchemaID:2, TableID:104, RowCount:0, ArgLen:4, start time: 2024-05-15 09:36:11.531 +0800 CST, Err:[ddl:8214]Cancelled DDL job, ErrCount:2, SnapshotVersion:0, LocalMode: false, UniqueWarnings:0"] [error="[ddl:8214]Cancelled DDL job"]
[2024/05/15 09:36:11.606 +08:00] [INFO] [ddl_worker.go:733] ["finish DDL job"] [worker="worker 18, tp add index"] [category=ddl] [jobID=106] [conn=136314882] [job="ID:106, Type:add index, State:cancelled, SchemaState:none, SchemaID:2, TableID:104, RowCount:0, ArgLen:4, start time: 2024-05-15 09:36:11.531 +0800 CST, Err:[ddl:8214]Cancelled DDL job, ErrCount:2, SnapshotVersion:0, LocalMode: false, UniqueWarnings:0"]

4. What is your TiDB version? (Required)

v8.0.0

commented

I know set the max-index-length configuration in one of tidb node is unreasonable. But i think the ddl worker can't panic at least. It might make more sense to print more directional information.

commented

Except for the scenario described by the issue. In addition, the index creation job was first executed in the background and changed the value of this configuration so that it would not pass the “sumLength > maxIndexLength” which is also a issue (although this is a very rare case).
https://github.com/pingcap/tidb/pull/50624/files#diff-45ad039a8b87835b475a52615355c444538d3a1f31a84d7b3ac94e15b1ce8721R126-R127