lock/lockForUpdate locks up all rows even combined with limit (Job Queue becomes unusable on large table)

[Jan Metzger]

Currently, when you create an SQL query with a limit, it gets translated into the following SQL like this:

\DB::table('laraveljobs')->first();

becomes:

select * from ( select * from laraveljobs ) where rownum = 1;

When you now add a lock request, I would expect the lock to be applied only on the limited rows. However, due to the execution order of the statement, the nested select gets executed first, including the lock request, locking up the full table before returning the first row.

\DB::table('laraveljobs')->lockForUpdate()->first();

becomes

select * from ( select * from laraveljobs for update ) where rownum = 1;

This behaviour kinda breaks the laravel database queue worker for a large pool of job entries. Our job table accumulated 40.000 jobs, but now the SQL statement of the laravel job worker takes almost 4 minutes to finish because it requests locks for all the rows instead of just for the first.

The code for getting the next job can be found in the Laravel Framework here: https://github.com/illuminate/queue/blob/master/DatabaseQueue.php#L236

    protected function getNextAvailableJob($queue)
    {
        $job = $this->database->table($this->table)
                    ->lock($this->getLockForPopping())
                    ->where('queue', $this->getQueue($queue))
                    ->where(function ($query) {
                        $this->isAvailable($query);
                        $this->isReservedButExpired($query);
                    })
                    ->orderBy('id', 'asc')
                    ->first();

        return $job ? new DatabaseJobRecord((object) $job) : null;
    }

I think the solution would be to make the lock part of the constraint parameter of compileTableExpression in OracleGrammar.php to add it behind the limit. Do you have an idea how to solve this issue?

https://github.com/yajra/laravel-oci8/blob/10.x/src/Oci8/Query/Grammars/OracleGrammar.php#L178

[Arjay Angeles]

No idea yet how to solve this.

I think the solution would be to make the lock part of the constraint parameter of compileTableExpression in OracleGrammar.php to add it behind the limit. Do you have an idea how to solve this issue?

What do you think is the proper SQL to be generated in this case? And Yes, I think we should adjust the grammar to take lock as part of the constraint.

Please do not hesitate to submit a PR if you can. Thanks!

[Arjay Angeles]

select * from (select * from laraveljobs) where rownum = 1 for update;

I think this is the SQL that you want to generate. Will check further if this works on plain SQL run when I got the chance.

[Jan Metzger]

Yes, that would be the query I think would be correct. I'll try to make a PR for the constraint change / tests soon. Thanks for the answer.