[Question] Is there a way to get safe autoincrementing keys?

[Daniel Mazurkiewicz]

Once I wrote separate lib which handled atomic operations on memory mapped file, but is there a simple easy way to get it directly from this lib?

[Kris Zyp]

node-lmdb doesn't have a dedicated function for this, but you can do this relatively simply by creating a transaction to increment and get the next key:

let txn = env.beginTxn(); // write transaction gives you an exclusive lock on the db
let id = txn.getNumber(db, 'next-id') || 1;
txn.putNumber(db, 'next-id', id + 1);
txn.commit();

Creating and committing a transaction solely for id incrementation may not be all that efficient though, you'd probably want to do this in the same transaction where you write data with your new id, if you are concerned about performance.

[Daniel Mazurkiewicz]

@kriszyp

let txn = env.beginTxn(); // write transaction gives you an exclusive lock on the db
let id = txn.getNumber(db, 'next-id') || 1;
txn.putNumber(db, 'next-id', id + 1);
txn.commit();

before writing, there is a reading command, is it really guaranteed not to run in parallel? (so no same readings from multiple forks of process???). Transaction at beginTxn() doesn't know yet it is gonna be write kind, then there is a read, then increment and then write, and at the end commit. So far I thought that commit is a moment to flush back data to database, and before that every read will end up with old value - thus other forks might assign same ID...

Ps. This is what I made and was using so far: https://github.com/DanielMazurkiewicz/atomic-counters

[Kris Zyp]

is it really guaranteed not to run in parallel? (so no same readings from multiple forks of process???). Transaction at beginTxn() doesn't know yet it is gonna be write kind

Yes. This transaction is a write transaction from the start; you have to use the readOnly option/flag to specify a read-only transaction (and read-only transactions can operate in parallel), and without that flag, it is a write transaction. Write transactions are absolutely guaranteed to be guarded with system-wide lock/mutex and all other processes (and threads) are blocked from starting a transaction on the database environment until the transaction is finished and that data is committed and consistently visible and available to other processes that are waiting to open a (write) transaction. LMDB guarantees that you can do this read and write as a true ACID operation with full atomicity and data consistency after completion (and durability as long as you don't disable sync flushes), which means you can definitely do this to get safely incremented ids without risk of overlap or race conditions.