tarantool / moonwalker

Iterate over one space with the following logic

Per-space updater for Tarantool 1.6+

A Lua module for Tarantool 1.6+ that allows iterating over one space with the following logic:

1. Phase #1 (сollect):

2. Create an iterator and iterate over the space for not more than pause items.

3. Put items-to-update into a temporary Lua table.

4. Yield the fiber, then reposition the iterator to GT(last selected tuple).

5. If collected enough (take) tuples, switch to phase #2 (update).

6. Phase #2 (update):

7. Iterate over the temporary table.

8. For each element, call the actor function.

9. Reposition the iterator to GT(last selected tuple) and switch back to phase #1 (collect).

Table of contents

• Parameters
• Examples

Parameters

• space - the space to process.
• index (optional) - the index to iterate by. If not defined, use the primary index.
• examine: (optional, function:boolean) - called during phase #1 (collect). Must not yield.
• actor: (function, altname: updater) - called during phase #2 (update) for every examined tuple.
• pause: 1000 (number) - make fiber.yield after stepping over this number of items.
• take: 600 (number) - how many items should be collected before switching to phase #2 (update).
• fp: 3 (number) - fiber pool. how many fibers in parallel would update tuples
• txn: false (boolean) - wrap update in transaction. Could improve the performance.
• dryrun: false (boolean) - don't call the actor, only print the statistics.
• silent: false (boolean) - don't do informational prints (useful for use in code)
• limit: 2^63 (optional, number) - process not more than this number of items. Useful for testing.
• progress: 2% (optional, string or number) - print a progress message every N records or percent.

Examples

local moonwalker = require 'moonwalker'

-- update the whole database (the simplest example)
moonwalker {
	space = box.space.users;
	actor = function(t)
		box.space.users:update({t[1]},{
			{'=', 2, os.time()}
		})
	end;
}

-- update the database, add missed fields (example with 'examine')
moonwalker {
	space = box.space.users;
	examine = function(t)
		return #t < 4; -- user tuple has only 3 fields
	end;
	actor = function(t)
		box.space.users:update({t[1]},{
			{'=', 4, "newfield"}
		})
	end;
}

-- iterate by a specific index
moonwalker {
	space = box.space.users;
	index = box.space.users.index.name; -- iterate over index 'name'
	pause = 100; -- be very polite, but slow: pause after every 100 records
	take  = 100; -- collect 100 items for update
	limit = 1000; -- stop after examining the first 1000 tuples
	examine = function(t)
		return #t < 4;
	end;
	actor = function(t)
		box.space.users:update({t[1]},{
			{'=',4,"newfield"}
		})
	end;
}