Parser support for `PARTITION BY`

Question

Parser support for `PARTITION BY`

xocasdashdash opened this issue a month ago · comments

Joaquín Fernández Campo commented a month ago

Hey everyone!
I'm using this project for running tests starting a mysql DB in memory. I'm currently hitting an issue when using PARTITION BY clauses in my table definitions.

I've checked and the vitess sql parser does support it (attaching an example), but the fork used here doesn't

Trying to parse this table works in vitess using the vitess.io/vitess/go/vt/sqlparser package

CREATE TABLE Test (
		a_key varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
		created_at datetime,
		PRIMARY KEY (a_key, created_at)
	) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
	PARTITION BY RANGE (MONTH(created_at));

But it fails with a syntax error when running it in go-mysql-server

I'm assuming that the fork is from before this vitessio/vitess#9712 PR was merged into vitess, but i'm wondering if there's any plan to increase support for parsing these kind of statements.

package main

import (
	"fmt"
	"log"
	"vitess.io/vitess/go/vt/sqlparser"
)

func main() {
	sql := `CREATE TABLE Test (
		a_key varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
		created_at datetime,
		PRIMARY KEY (a_key, created_at)
	) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
	PARTITION BY RANGE (MONTH(created_at));`
	p := sqlparser.Parser{}
	stmt, err := p.Parse(sql)
	if err != nil {
		log.Fatalf("Error parsing SQL: %v", err)
	}

	// Asserting the parsed statement is a CreateTable statement
	createTable, ok := stmt.(*sqlparser.CreateTable)
	if !ok {
		log.Fatalf("Parsed query is not a CREATE TABLE statement")
	}

	fmt.Println("Table Name:", sqlparser.String(createTable.Table.Name))
	fmt.Println("Columns:")
	for _, col := range createTable.TableSpec.Columns {
		fmt.Printf("  - %s %s\n", sqlparser.String(col.Name), sqlparser.String(col.Type))
	}
	if createTable.TableSpec.PartitionOption != nil {
		fmt.Println("Partitioning Info:", sqlparser.String(createTable.TableSpec.PartitionOption))
	}

	sql2 := `ALTER TABLE Test MODIFY b_key VARCHAR(20) NOT NULL, ALGORITHM=INPLACE, LOCK=NONE;`

	stmt, err = p.Parse(sql2)
	if err != nil {
		log.Fatalf("Error parsing SQL: %v", err)
	}

	alterTable, ok := stmt.(*sqlparser.AlterTable)
	if !ok {
		log.Fatalf("Parsed query is not a CREATE TABLE statement")
	}

	fmt.Printf("%#v", alterTable)
	fmt.Println("Table Name:", sqlparser.String(alterTable.Table))
	fmt.Println("Alter Options:")
	for _, opt := range alterTable.AlterOptions {
		fmt.Printf("  - %s %s\n", sqlparser.String(opt), ))
	}
	if alterTable.PartitionOption != nil {
		fmt.Println("Partitioning Info:", sqlparser.String(createTable.TableSpec.PartitionOption))
	}

}

Tim Sehn · Answer 1 · Thu Apr 25 2024 04:51:39 GMT+0800 (China Standard Time)

We can probably easily make the syntax a no-op. Would that work for you?

Joaquín Fernández Campo · Answer 2 · Thu Apr 25 2024 04:59:45 GMT+0800 (China Standard Time)

Yes! That would be good!

These statements run as part of our migrations so it's needed for us.

I think there was another example of a MySQL partition query failing. I'll update the issue tomorrow

Joaquín Fernández Campo · Answer 3 · Thu Apr 25 2024 16:16:38 GMT+0800 (China Standard Time)

ok the other statement that we have issues with is when you specify the algorithm when a table is modified. In the example i shared would be sql2

and this kind of statement:

ALTER TABLE Test MODIFY b_key VARCHAR(20) NOT NULL, ALGORITHM=INPLACE, LOCK=NONE;

James Cor · Answer 4 · Fri Apr 26 2024 02:11:46 GMT+0800 (China Standard Time)

Hey @xocasdashdash thanks for reporting this issue.

This PR, dolthub/vitess#340, should properly parse the create table ... query you mentioned.
It's currently in review, and should be merged soon.

Heads up, the create table ... query parses in MySQL, but throws

ERROR: 1492: For RANGE partitions each partition must be defined

It'll still be a no-op in dolt/gms right now, but may eventually change to match MySQL.

I'm currently looking into adding parser support for the alter table ... query.

Joaquín Fernández Campo · Answer 5 · Fri Apr 26 2024 14:25:01 GMT+0800 (China Standard Time)

Hey @xocasdashdash thanks for reporting this issue.

This PR, dolthub/vitess#340, should properly parse the create table ... query you mentioned. It's currently in review, and should be merged soon.

Heads up, the create table ... query parses in MySQL, but throws
ERROR: 1492: For RANGE partitions each partition must be defined
It'll still be a no-op in dolt/gms right now, but may eventually change to match MySQL.

I'm currently looking into adding parser support for the alter table ... query.

Ahh thanks so much, yeah we actually have the full range defined.

Here's an example:

CREATE TABLE `t` (
   `t_key` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
   `created_at` datetime,
   PRIMARY KEY (`t_key`, `created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
	PARTITION BY RANGE (MONTH(`created_at`)) (
		PARTITION p1 VALUES LESS THAN (2),
        PARTITION p2 VALUES LESS THAN (3),
        PARTITION p3 VALUES LESS THAN (4),
        PARTITION p4 VALUES LESS THAN (5),
        PARTITION p5 VALUES LESS THAN (6),
        PARTITION p6 VALUES LESS THAN (7),
        PARTITION p7 VALUES LESS THAN (8),
        PARTITION p8 VALUES LESS THAN (9),
        PARTITION p9 VALUES LESS THAN (10),
        PARTITION p10 VALUES LESS THAN (11),
        PARTITION p11 VALUES LESS THAN (12),
        PARTITION p12 VALUES LESS THAN (13)
	);
	```
	
Is this going to be supported too?

James Cor · Answer 6 · Fri Apr 26 2024 14:56:41 GMT+0800 (China Standard Time)

Yep, that query is already supported in dolt/gms right now!
It is a no-op though

tmp/main*> CREATE TABLE `t` (
        ->     `t_key` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
        ->     `created_at` datetime,
        ->     PRIMARY KEY (`t_key`, `created_at`)
        -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
        -> PARTITION BY RANGE (MONTH(`created_at`)) (
        ->     PARTITION p1 VALUES LESS THAN (2),
        ->     PARTITION p2 VALUES LESS THAN (3),
        ->     PARTITION p3 VALUES LESS THAN (4),
        ->     PARTITION p4 VALUES LESS THAN (5),
        ->     PARTITION p5 VALUES LESS THAN (6),
        ->     PARTITION p6 VALUES LESS THAN (7),
        ->     PARTITION p7 VALUES LESS THAN (8),
        ->     PARTITION p8 VALUES LESS THAN (9),
        ->     PARTITION p9 VALUES LESS THAN (10),
        ->     PARTITION p10 VALUES LESS THAN (11),
        ->     PARTITION p11 VALUES LESS THAN (12),
        ->     PARTITION p12 VALUES LESS THAN (13)
        -> );
tmp/main*> show create table t;
+-------+--------------------------------------------------------------------+
| Table | Create Table                                                       |
+-------+--------------------------------------------------------------------+
| t     | CREATE TABLE `t` (                                                 |
|       |   `t_key` varchar(100) NOT NULL,                                   |
|       |   `created_at` datetime NOT NULL,                                  |
|       |   PRIMARY KEY (`t_key`,`created_at`)                               |
|       | ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci |
+-------+--------------------------------------------------------------------+
1 row in set (0.00 sec)

I'm still working on getting the ALTER TABLE... statement parsing.