Self-referencing foreign key constraint breaks auto-incrementing ids in memory mode

[Mateya Lucic]

Related to #2349

I receive a

Error 1062 (HY000): duplicate primary key given: [2]

error when doing these queries in memory mode

CREATE TABLE table1 (
	id int NOT NULL AUTO_INCREMENT,
	name text,
	parentId int DEFAULT NULL,
	PRIMARY KEY (id),
	CONSTRAINT myConstraint FOREIGN KEY (parentId) REFERENCES table1 (id) ON DELETE CASCADE
)

INSERT INTO table1 (name, parentId) VALUES ('tbl1 row 1', NULL);
INSERT INTO table1 (name, parentId) VALUES ('tbl1 row 2', 1);
INSERT INTO table1 (name, parentId) VALUES ('tbl1 row 3', NULL);

If I remove the CONSTRAINT myConstraint FOREIGN KEY (parentId) REFERENCES table1 (id) I don't get any errors.

Full go code reproduction:

package main

import (
	"context"
	"database/sql"
	"fmt"

	sqle "github.com/dolthub/go-mysql-server"
	gmsSql "github.com/dolthub/go-mysql-server/sql"
	msql "github.com/dolthub/go-mysql-server/sql"
	"github.com/dolthub/go-mysql-server/sql/mysql_db"
	vsql "github.com/dolthub/vitess/go/mysql"

	"github.com/dolthub/go-mysql-server/memory"
	"github.com/dolthub/go-mysql-server/server"

	_ "github.com/go-sql-driver/mysql"
)

var (
	dbName  = "mydb"
	address = "localhost"
	port    = 3306
)

func TestBadAutoIncrement() {
	mdb := memory.NewDatabase(dbName)
	mdb.EnablePrimaryKeyIndexes()
	pro := memory.NewDBProvider(mdb)
	engine := sqle.NewDefault(pro)

	config := server.Config{
		Protocol: "tcp",
		Address:  fmt.Sprintf("%s:%d", address, port),
	}
	sessionBuilder := func(ctx context.Context, c *vsql.Conn, addr string) (gmsSql.Session, error) {
		host := ""
		user := ""
		mysqlConnectionUser, ok := c.UserData.(mysql_db.MysqlConnectionUser)
		if ok {
			host = mysqlConnectionUser.Host
			user = mysqlConnectionUser.User
		}
		client := gmsSql.Client{Address: host, User: user, Capabilities: c.Capabilities}
		return memory.NewSession(msql.NewBaseSessionWithClientServer(addr, client, c.ConnectionID), pro), nil
	}
	s, err := server.NewServer(config, engine, sessionBuilder, nil)
	if err != nil {
		panic(err)
	}
	go func() {
		if err = s.Start(); err != nil {
			panic(err)
		}
	}()

	db, err := sql.Open("mysql", "/mydb")
	if err != nil {
		panic(err)
	}
	_, err = db.Exec(`
CREATE TABLE table1 (
	id int NOT NULL AUTO_INCREMENT,
	name text,
	parentId int DEFAULT NULL,
	PRIMARY KEY (id),
	CONSTRAINT myConstraint FOREIGN KEY (parentId) REFERENCES table1 (id) ON DELETE CASCADE
)
	`)
	if err != nil {
		panic(err)
	}
	_, err = db.Exec("INSERT INTO table1 (name, parentId) VALUES ('tbl1 row 1', NULL)")
	if err != nil {
		panic(err)
	}
	_, err = db.Exec("INSERT INTO table1 (name, parentId) VALUES ('tbl1 row 2', 1)")
	if err != nil {
		panic(err)
	}
	_, err = db.Exec("INSERT INTO table1 (name, parentId) VALUES ('tbl1 row 3', NULL)")
	if err != nil {
		panic(err)
	}
}

It seems like the issue from #2349 is still present when making self-referential foreign key constraints

[Mateya Lucic]

CC @max-hoffman since you implemented the fix for the related issue

[Maximilian Hoffman]

I can repro, this one is a little trickier but basically the same thing. Before we were bleeding information between transactions, failing to reset editor data created an insert editor with a split brain. Now we're bleeding information between analysis and execution. When we resolve foreign keys the session initializes a tableEditor for the FK table. Because the FK table is the same as the insert table, we lose track of the difference when we initialize the execution insert editor. Same error root cause, the editor that lands id increments is not the editor with the insert data.

I do not understand why the FK table editor needs to share the same namespace as the insert table and will look into this. But the solution is either 1) make the session namespace differentiate these two states, 2) make sure the insert editor tracks the analysis editor, 3) make some structural change where FKs do what they need to do in a different way. I guess it depends on whether we can take shortcuts with DELETEs and FK CASCADEs accumulating in the same editor, or if those have to be applied sequentially in different working sets.