issues with multiple transactions

[cvlmtg]

Hi, I've a problem when my server receives multiple requests which leads to a certain transaction executed in parallel (according to the async module definition of "parallel", i.e. a transaction is started before the previous one is completed).

my transaction is done this way (with pseudo-coffeescript):

doStuff = (db, done) =>
  @readFromDb userId, =>
    @doSomethingElse userId, =>
      child = Child.create
        userId: userId

      @addChild(child).classic done
  return

@db.transaction(doStuff).classic callback

the problem is that sometimes the done callback is undefined. moreover looking at mysql query log it seems that the connection pool is used in some strange way, because e.g. I see two insert which should belong to two different transaction coming from the same connection id.

when in my unit test I send the requests to the server with "async.eachSeries" everything works, but if I use "async.each" then the problem arises.
I'm not sure if this is a bug or I simply have dome something wrong in my code, but I can't spot any obvious error in my transaction function...

thanks in advance for your help.

[Doug Martin]

The issue is that node is single threaded, so in when patio begins a transaction and you require done it means that the transaction is not complete until done is called, when you do a query an asynchronous action within the transaction and start another one it assumes that the query also belongs in the same transaction.

So in this case because you have start a transaction and perform another one it will be included in the first transaction started. I realize this is a little confusing but because of the async nature of node it was the only way I could think of doing it while conforming to the async nature of node.js APIs.

The way we use it at C2FO is we build up our our saves, updates etc and perform them as a single action within the transaction, using multiInsert or db.run with a block of SQL.

For example

function doStuff(){
    return readFromDb().chain(function(models){
        return db.transaction(function(){
             return addChild(Child.create({userId: userId});
         });
     });
};

doStuff();

I hope this helps, we are open to suggestions on how this might be improved.

[cvlmtg]

Hi, thanks for the answer. I am still digging in patio source and I think I'm starting to understand what you say. I saw the __transactionProxy method and yes, I agree it's a bit confusing but not a simple task to solve...

unfortunately your example isn't valid in my application, because even the mock readFromDb() method must be run inside the transaction (e.g. I need to check that children are less than N before adding another one)

I was starting to tinker with the idea of adding some option to say "this transaction() call starts a new transaction even if one is still running", but I don't know yet if it's a dead end... after all how can subsequent commands says to which transaction they belong?
I'll see if I can make it work and let you know.

[Doug Martin]

Ok after some thought about how we typically use transactions, I came up with a solution that I think will work quite nicely, but of course if you have any suggestion feel free to comment!

So in v0.5.0 transactions now have a new option that can be passed in called isolate with will cause the transaction to be guaranteed to run separately from other transactions. The reason I went this route is because when writing code we typically have a couple of entry points to code that we want to ensure are in their own transaction i.e. when a new service call is made or a crud operation is performed.

An example might look like...

db.transaction({isolate: true}, function(){
   ///do some work!
});

[cvlmtg]

hi, thanks for the support!

I'll try this new version and see if it works with our code. The only suggestion I have at the moment is if it would it be possible to specify the isolate option once for all (e.g. on createConnection() options) to avoid specifying it on every transaction.

btw I've given a quick look on the commit and inside __enqueueTransaction you do a return ret.promise(), which is assigned to the variable ret (line 575 of lib/database/query.js) and two lines below you do a return ret.promise(). In other words you do a return ret.promise().promise(). is this correct?

[cvlmtg]

ok, I've started playing with the new code and I've a couple of thing to report:

1. forget my suggestion about the global option, I think I can be fine with specifying isolate on every transaction. As it's basically the opposite of the savepoint options, having both explicitly put in the code might be slightly better for code clarity.

2. on line 607 of lib/database/query.js you still have

if (this.supportsSavepoints && !isUndefinedOrNull(arg1.savepoint))

which means that {savepoint:false} will start a savepoint... I think this is wrong.

thanks again for the support!

[cvlmtg]

ok I think I've found an issue, I'll open a new one as this one is closed ;)