boltdb / bolt

An embedded key/value database for Go.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

page already freed on certain builds

tehsphinx opened this issue · comments

Sorry to bring up another issue with this headline... I already checked the other two issues and do not have the impression that they match my case.

Here my error

panic: page 4 already freed

goroutine 416 [running]:
.../vendor/github.com/boltdb/bolt.(*freelist).free(0x202f7ec0, 0x5, 0x0, 0x3014000)
/ext-go/1/src/.../vendor/github.com/boltdb/bolt/freelist.go:121 +0x263
.../vendor/github.com/boltdb/bolt.(*node).spill(0x20356d40, 0x2064bc64, 0x6)
/ext-go/1/src/.../vendor/github.com/boltdb/bolt/node.go:363 +0x19f
.../vendor/github.com/boltdb/bolt.(*Bucket).spill(0x2032eb0c, 0xe, 0x38554104)
/ext-go/1/src/.../vendor/github.com/boltdb/bolt/bucket.go:570 +0x13d
.../vendor/github.com/boltdb/bolt.(*Tx).Commit(0x2032eb00, 0x0, 0x0)
/ext-go/1/src/.../vendor/github.com/boltdb/bolt/tx.go:163 +0xff
.../vendor/github.com/boltdb/bolt.(*DB).Update(0x20244fc0, 0x2056de80, 0x0, 0x0)
/ext-go/1/src/.../vendor/github.com/boltdb/bolt/db.go:605 +0xc9

Here my Code snippet:

// Add adds a value to the db using NextSequence to autoincrement the key
func (s *DB) Add(tableName string, value []byte) (uint64, error) {
	var id uint64
	err := s.Conn.Update(func(tx *bolt.Tx) error {
		bt, err := tx.CreateBucketIfNotExists([]byte(tableName))
		if err != nil {
			return err
		}

		id, err = bt.NextSequence()
		if err != nil {
			return err
		}

		return bt.Put(itob(id), value)
	})
	return id, errors.WithStack(err)
}

// itob returns an 8-byte big endian representation of v.
func itob(i uint64) []byte {
	b := make([]byte, 8)
	binary.BigEndian.PutUint64(b, i)
	return b
}

The error occurs the second time this code is executed (every time).

The code runs fine on MacOS (amd64) and Windows (386) without cgo. Disabling cgo disables a few packages that use windows dlls.
When compiling the final version with cgo (and dlls) using xgo --targets=windows/386 . the resulting windows executable will show the above issue.

I added some logs to the boltdb code and checked which pages are being freed. On every call the same pages are being freed. The node.pgid on node.go:363 is also the same on every call (calls are several secods apart). Why?

Since in one of the other issues on this topic there is talk about race conditions I made sure the entire connection is used by a single goroutine using the action pattern. This did not resolve the issue either:

// Add adds a value to the db using NextSequence to autoincrement the key
func (s *DB) Add(tableName string, value []byte) (uint64, error) {
	// added this to make sure there is no other goroutine using the byte array at the same time
	var valCopy = make([]byte, 0, len(value))
	valCopy = append(valCopy, value...)

	var id uint64
	chErr := make(chan error)
	s.Run(s.Action)
	s.Action <- func() {
		err := s.Conn.Update(func(tx *bolt.Tx) error {
			bt, err := tx.CreateBucketIfNotExists([]byte(tableName))
			if err != nil {
				return err
			}

			id, err = bt.NextSequence()
			if err != nil {
				return err
			}

			return bt.Put(itob(id), valCopy)
		})
		chErr <- err
	}
	err := <-chErr
	return id, errors.WithStack(err)
}

Anyone any idea on how to solve this issue?