Giters

dolthub / go-mysql-server

A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Creating an enum column with collation utf8_czech_ci causes panic

mibk opened this issue · comments

commented

I'm using go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6.
To reproduce, use the in-memory example server, and just:

CREATE TABLE `foo` (
  `bar` enum('Y','N') COLLATE utf8_czech_ci NOT NULL
) ENGINE=InnoDB;

using the mysql client connected to the server.

It appears getRuneWeight is nil in sql/collations.go:WriteWeightString:

		getRuneWeight := collationArray[c].Sorter
		i := 0
		buf := *weightBuffers.Get().(*[]byte)
		if cap(buf) < len(str)*4 {
			buf = make([]byte, len(str)*4)
		}
		for len(str) > 0 {
			// All strings (should) have been decoded at this point, so we can rely on Go's internal string encoding
			runeFromString, strRead := utf8.DecodeRuneInString(str)
			if strRead == 0 || strRead == utf8.RuneError {
				return ErrCollationMalformedString.New("hashing")
			}
			runeWeight := getRuneWeight(runeFromString)
			buf[i*4] = byte(runeWeight)
			buf[i*4+1] = byte(runeWeight >> 8)
			buf[i*4+2] = byte(runeWeight >> 16)
			buf[i*4+3] = byte(runeWeight >> 24)
			_, err := hash.Write(buf[i*4 : i*4+4])
			if err != nil {
				return err
			}
			str = str[strRead:]
			i++
		}
		weightBuffers.Put(&buf)
See the full stack trace here 
time="2023-05-29T13:31:57+02:00" level=error msg="mysql_server caught panic:
runtime error: invalid memory address or nil pointer dereference
/Users/u/.local/opt/go-v1.20.4/src/runtime/panic.go:260 (0x104acbc)
\tpanicmem: panic(memoryError)
/Users/u/.local/opt/go-v1.20.4/src/runtime/signal_unix.go:837 (0x104ac8c)
\tsigpanic: panicmem()
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/collations.go:884 (0x14a2605)
\tcom/dolthub/go-mysql-server/sql.CollationID.WriteWeightString: runeWeight := getRuneWeight(runeFromString)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/collations.go:906 (0x14a28d8)
\tcom/dolthub/go-mysql-server/sql.CollationID.HashToUint: err := c.WriteWeightString(hash, str)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/types/enum.go:79 (0x157a7f4)
\tcom/dolthub/go-mysql-server/sql/types.CreateEnumType: hashedVal, err := collation.HashToUint(value)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/types/conversion.go:345 (0x1573936)
\tcom/dolthub/go-mysql-server/sql/types.ColumnTypeToType: return CreateEnumType(ct.EnumValues, collation)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:2898 (0x1a6b053)
\tcom/dolthub/go-mysql-server/sql/parse.columnDefinitionToColumn: internalTyp, err := types.ColumnTypeToType(&cd.Type)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:2881 (0x1a6acee)
\tcom/dolthub/go-mysql-server/sql/parse.TableSpecToSchema: column, err := columnDefinitionToColumn(ctx, cd, tableSpec.Indexes)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:2447 (0x1a67153)
\tcom/dolthub/go-mysql-server/sql/parse.convertCreateTable: schema, collation, err := TableSpecToSchema(ctx, c.TableSpec, false)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:1270 (0x1a5d128)
\tcom/dolthub/go-mysql-server/sql/parse.convertDDL: return convertCreateTable(ctx, c)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:172 (0x1a52f16)
\tcom/dolthub/go-mysql-server/sql/parse.convert: return convertDDL(ctx, query, n, false)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:118 (0x1a52b8e)
\tcom/dolthub/go-mysql-server/sql/parse.parse: node, err := convert(ctx, stmt, s)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/sql/parse/parse.go:77 (0x1be7004)
\tcom/dolthub/go-mysql-server/sql/parse.ParseOne: return parse(ctx, query, true)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/server/handler.go:310 (0x1be6fe9)
\tcom/dolthub/go-mysql-server/server.(*Handler).doQuery: parsed, prequery, remainder, _ = parse.ParseOne(ctx, query)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/server/handler.go:577 (0x1be98a7)
\tcom/dolthub/go-mysql-server/server.(*Handler).errorWrappedDoQuery: remainder, err := h.doQuery(c, query, mode, bindings, callback)
/Users/u/pkg/mod/github.com/dolthub/go-mysql-server@v0.15.1-0.20230525225634-4047c40864f6/server/handler.go:164 (0x1be55c8)
\tcom/dolthub/go-mysql-server/server.(*Handler).ComMultiQuery: return h.errorWrappedDoQuery(c, query, MultiStmtModeOn, nil, callback)
/Users/u/pkg/mod/github.com/dolthub/vitess@v0.0.0-20230518175744-593ce40efc46/go/mysql/conn.go:1447 (0x145fbf1)
\tcom/dolthub/vitess/go/mysql.(*Conn).execQuery: remainder, err = handler.ComMultiQuery(c, query, resultsCB)
/Users/u/pkg/mod/github.com/dolthub/vitess@v0.0.0-20230518175744-593ce40efc46/go/mysql/conn.go:972 (0x145b404)
\tcom/dolthub/vitess/go/mysql.(*Conn).handleNextCommand: for query, err = c.execQuery(query, handler, multiStatements); err == nil && query != \"\"; {
/Users/u/pkg/mod/github.com/dolthub/vitess@v0.0.0-20230518175744-593ce40efc46/go/mysql/server.go:484 (0x14764e7)
\tcom/dolthub/vitess/go/mysql.(*Listener).handle: err := c.handleNextCommand(l.handler)
/Users/u/.local/opt/go-v1.20.4/src/runtime/asm_amd64.s:1598 (0x1066b00)
\tgoexit: BYTE\t$0x90\t// NOP
"
commented
commented

OK, so the real issue is that it's just not supported yet. I believe it is a valid collation but is just an alias for utf8mb3_czech_ci, which seems to be indicated here in sql/collations.go:

	Collation_utf8_turkish_ci          = Collation_utf8mb3_turkish_ci
	Collation_utf8_czech_ci            = Collation_utf8mb3_czech_ci
	Collation_utf8_danish_ci           = Collation_utf8mb3_danish_ci
commented

@Hydrocharged will fix the panic and get the collation supported today. I'm not sure if he considered collation aliases, hence the panic.

commented

Collation aliases are fully supported. The panic came from an oversight with enum and set, as those use the collation while creating the type, which occurs before we check if the collation has been implemented. I added a few more collation checks, so now we'll return the proper error (which just says to let us know if you need us to implement the collation).

The linked PR includes a fix for the panic, and it adds the missing collation as well.