SQL Mapper library is causing deadlocks on updates

[Elie Obeid]

We have multiple tables foo and bar, InnoDB schema, MySQL 5.7 (Aws RDS)

If you have many requests calling

$foo->update();
$bar->update();

then $bar->update(); might cause a deadlock

Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction 
[/var/www/vendor/bcosca/fatfree/lib/db/sql.php:230]PHP message: [vendor/bcosca/fatfree/lib/db/sql.php:230] 
PDOStatement->execute()PHP message: [vendor/bcosca/fatfree/lib/db/sql/mapper.php:542] 
DB\SQL->exec(['UPDATE `bar` SET `x`=?,`x` ...])

I believe the reason is when you call update(), then mapper.php

$this->db->exec($sql,$args);

that is called, on line 542, which then calls

$this->commit();
return $result;

in sql.php line 273.

Therefore every update statement triggers a transaction, and one is getting delayed until the previous finishes.

And we only get return $result from sql.php, so we can't do something like

try {
    // First of all, let's begin a transaction
    $db->beginTransaction();
    
    // A set of queries; if one fails, an exception should be thrown
    $db->query('first query');
    // If we arrive here, it means that no exception was thrown
    // i.e. no query has failed, and we can commit the transaction
    $db->commit();
} catch (\Throwable $e) {
    // An exception has been thrown
    // We must rollback the transaction
    $db->rollback();
    throw $e; // but the error must be handled anyway
}

because we don't know the result of the commit.

Is there anyway to address this issue while using f3 mapper?

[Christian Knuth]

Well, actually I would think that you can start a new transaction right before the mapper->update call. If the table is that large that an update might lead to a deadlock, try adding the appropriate indexes on the fields your're filtering on in the update query.

[Elie Obeid]

@ikkez

We do updates like this

$this->bar->getWhere("id = '$x'");
$this->bar->x=1;
$this->bar->y=2;
$this->bar->z=3;
$this->bar->update();

id is a primary key, so it's indexed, x, y, and z are not. is there any other way to do the updates?

[Chris]

I don't see any application-caused reason for a deadlock here, maybe there is some other action happening in the tables. Do those two tables interfere somehow? Do they have foreign keys that could be involved? Are there any other simultaneous processes active on the table like historization of records with moving them to other tables?
Try to investigate the real cause of the lock using SHOW ENGINE InnoDB STATUS

Afaik, you can control the auto-commit-behaviour so that you are able to bundle both updates in one commit (they share the same transaction)

Here is a good explanation what could cause a deadlock: https://dba.stackexchange.com/questions/210949/why-does-this-query-result-in-deadlock/211328#211328

[Elie Obeid]

@KOTRET SHOW ENGINE InnoDB STATUS only helps if the deadlock happens while you're monitoring it right? so you have to wait for like two or 3 hours and check periodically if the deadlock happened, correct?

the link you provided is useful but doesn't explain how such solutions can be implemented in f3 framework. One solution would be to stop using the SQL mapper and write PDO, but other than that, we can't do anything right? the mapper limits us to getwhere and those generic functions.

[Florent]

@elieobeid7 you seem to think that we cannot catch SQL-related exceptions, but we can.

DB\SQL is a subclass of PDO so it can throw catchable PDO exceptions.

The trick is that those are disabled by default, so you need to enable them first:

$db = new DB\SQL($dsn, $user, $pwd, [\PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION]);

Once PDO exceptions are enabled, you can write such things:

try {
    // begin transaction
    $db->begin();
    // execute one or more SQL queries such as:
    $db->exec('UPDATE mytable SET foo=123 WHERE id=456');
    // or:
    $mapper->save();
    // then commit transaction
    $db->commit();
} catch (\PDOException $e) {
    $err=$e->errorInfo;
  //$err[0] contains the error code (e.g: 23000)
  //$err[2] contains the driver specific error message (e.g: PRIMARY KEY must be unique)
}

NB: no need to rollback the transaction in the catch section, this is automatic.

[Elie Obeid]

@xfra35 thank you, I'll try that and let you know

[Elie Obeid]

@xfra35 do I really need to do a transaction like that? because $db->exec() will call exec function in vendor\bcosca\fatfree\lib\db\sql.php line 152, which would also begin a transaction on line 166, doesn't that mean we're doing two transactions?

[PJH]

which would also begin a transaction on line 166

Only if $this->trans is false (line 165.) Which it wouldn't be because the $db->begin() before the $db->exec() would set it to true.

So, arguably, the $db->begin() there is simply redundant.

[Elie Obeid]

ok thank you for the clarification. I'll close the issue then.

[Florent]

When you call a single $db->exec(), then it includes an automatic transaction in the background.

So, the two following snippets are identical:

// 1) automatic transaction
$db->exec('UPDATE mytable etc.'); // or ['UPDATE mytable1 etc.', 'UPDATE mytable2 etc.']
// 2) manual transaction
$db->begin();
$db->exec('UPDATE mytable etc.'); // or ['UPDATE mytable1 etc.', 'UPDATE mytable2 etc.']
$db->commit();

But the two following snippets are not identical:

// 1) automatic transactions (2 distinct transactions)
$db->exec('UPDATE mytable1 etc.'); // transaction 1
$db->exec('UPDATE mytable2 etc.'); // transaction 2
// 2) manual transaction (1 single transaction)
$db->begin();
$db->exec('UPDATE mytable1 etc.');
$db->exec('UPDATE mytable2 etc.');
$db->commit();

In my initial example, I manually started the transaction in order to encompass several SQL queries in it (such as the $db->exec() and the $mapper->save().

[Elie Obeid]

Thank you for the clarification