Add meta to processes in groups
cjimison opened this issue · comments
It would be nice if I could have a group with a set of keys inside of it. For example: I have a collection of processes that belong to a group. However I need to make sure that certain requests always goes to a specific process within the group.
Group -> [{Key, Pid}, {Key2, Pid2}]
This way I can hash the requester ID to one of these Pid's in the group. Right now I have to pull all the processes out of the group and then pull key for each process, then do my mapping.
Example Elixir code: If Erlang is needed I can rewrite it
def processForKey(key) do
# Basic list comprehension to get all the pids with their keys (keys are integers)
members = for pid <- :syn.get_members(:mygroup) do
pidKey = :syn.find_by_pid(pid)
{pidKey, pid}
end
# sort the keys
members = Enum.sort(fn({k1, _}, {k2, _}) -> k1 > k2 end)
# pull out the process that should map to the passed in key
{_, pid} = Enum.at(members, rem(:erlang.phash2(key), length(members)))
pid
end
It would be nice if when adding the group I could add a meta/key value that I can pull out with a call like
:syn.get_members(:my group, :with_meta)
Thanks!
Hello!
I don't quite understand your use case: you're both registering Pids with their own name, and then adding those Pids to groups. Are you suggesting to add metadata to pids in a group, or a function that retrieves their registration name (if they have previously been registered)?
Use case aside, I think that adding metadata to processes might be interesting. I am also considering adding metadata to a group itself, however I want to keep Syn as simple as possible until a clear need emerges.
Would you help me understand better your use case?
Sure. So I have a Transaction System with a bunch of processes and I want to make sure I can pass a request to read/modify a users data. It is important that the same process does this work because it is for a game and order really matters to us. (Example: I got 10 gold, then two people try to steals 10 gold from me at the same time. One should succeed and one should fail, and since we use riak I need to order these so they both don't succeed.). So what I really want is a group of these "transaction" processes where I can access them in a order list.
myGroup = [TransPid123, TransPid234, TransPid142]
Then I can map my User Key to one of these 3 processes and make sure the mapping stays consistent across nodes (well as best one can in a distributed server system), without having to send a message to the trans Pids.
userKeyA = 72384738278
index into myGroup list = rem(userKey, length of myGroup)
And I can do this on any Node and know that a request to read/modify userKeyA data will map to TransPid234.
What I would like to do is as follows (I can put this in a Pull Request but honestly I don't know enough about syn yet to suggest coding changes :P )
In syn.hrl
-record(syn_groups_table, {
name = undefined :: any(),
pid = undefined :: undefined | pid() | atom(),
node = undefined :: atom(),
meta = undefined :: any()
}).
In syn.erl
-spec join(Name :: any(), Pid :: pid(), Meta :: term()) -> ok | {error, pid_already_in_group}.
join(Name, Pid, Meta) ->
syn_groups:join(Name, Pid, Meta).
-spec get_members_with_meta(Name :: any()) -> [{pid(), any()}].
get_members_with_meta(Name) ->
syn_groups:get_members_with_meta(Name).
In syn_groups.erl
-spec join(Name :: any(), Pid :: pid()) -> ok | {error, pid_already_in_group}.
join(Name, Pid) ->
Node = node(Pid),
gen_server:call({?MODULE, Node}, {join, Name, Pid, undefined}).
-spec join(Name :: any(), Pid :: pid(), Meta::any()) -> ok | {error, pid_already_in_group}.
join(Name, Pid, Meta) ->
Node = node(Pid),
gen_server:call({?MODULE, Node}, {join, Name, Pid, Meta}).
-spec get_members_with_meta(Name :: any()) -> [{pid(), any()}].
get_members_with_meta(Name) ->
i_get_members_with_meta(Name).
handle_call({join, Name, Pid, Meta}, _From, State) ->
case i_member(Pid, Name) of
false ->
%% add to table
mnesia:dirty_write(#syn_groups_table{
name = Name,
pid = Pid,
node = node(),
meta = Meta
}),
%% link
erlang:link(Pid),
%% return
{reply, ok, State};
_ ->
{reply, pid_already_in_group, State}
end;
-spec i_get_members_with_meta(Name :: any()) -> [Process :: #syn_groups_table{}].
i_get_members_with_meta(Name) ->
Processes = mnesia:dirty_read(syn_groups_table, Name),
lists:map(fun(Process) ->
{ Process#syn_groups_table.pid,
Process#syn_groups_table.meta}
end, Processes).
Thanks!
Thank you for your feedback @cjimison. I will get back to you on all the rest, but let me clarify that (in case you are) you shouldn't count on the order of member pids in the array returned by syn:get_members/1. The order is not guaranteed to match the order of joins, nor to be the same in successive calls.
Syn being eventually consistent, ensuring a proper join order would require a mechanism of vclocks or similar, which are not implemented in Syn.
Updated the readme with 3262bc7.
I think I may have found a simple way to guarantee the order in which members are returned. Would that matter to you?
Responses bellow:
On Mar 24, 2016, at 11:51 AM, Roberto Ostinelli notifications@github.com wrote:
Thank you for your feedback @cjimison https://github.com/cjimison. I will get back to you on all the rest, but let me clarify that (in case you are) you shouldn't count on the order of member pids in the array returned by syn:get_members/1. The order is not guaranteed to match the order of joins, nor to be the same in successive calls.
Yep. Which is why I wanted to assign a piece of meta data that I can use to order the list.
Syn being eventually consistent, ensuring a proper join order would require a mechanism of vclocks or similar, which are not implemented in Syn.
Since we control the meta value assigned to the group/pid I can sort it according and get something close enough (since these processes don’t come and go very often). The meta values we use are unique per node/cluster and we do use the vclock returned from Riak to further resolve consistency issues.
Updated the readme with 3262bc7 3262bc7.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #7 (comment)
That works too. Both would be awesome for sure :)
-Chris
On Mar 24, 2016, at 11:59 AM, Roberto Ostinelli notifications@github.com wrote:
I think I may have found a simple way to guarantee the order in which members are returned. Would that matter to you?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #7 (comment)
@cjimison this is now in master. Can you please take a look and provide feedback?
Thank you!