Slow query found and its fix

[superdioz]

We identified a slow query on MariaDB:

SELECT
MAX(p.post_modified_gmt) as last_modified
FROM wp_posts as p
WHERE 1 = 1 AND
p.post_status = 'publish'
AND
( p.ID IN
(
SELECT tr.object_id
FROM wp_term_relationships as tr
JOIN wp_term_taxonomy as tt ON tr.term_taxonomy_id = tt.term_taxonomy_id
WHERE tt.term_id = '7803'
)
)

The query is generated within the sitemap code (app/Common/Sitemap), in particoular in the following files/lines:

1. Root.php:353
2. Root.php:537
3. Content.php:365

The slowness is caused by the unknown type (or type mismatch) for the subquery, the fix is simple:
SELECT tr.object_id
has to become:
SELECT CAST(tr.object_id as int)

Please apply the fix the source so we don't have to overwrite it every plugin updates.
Thank you.

[Arnaud Broes]

Hey @superdioz,

I'm not entirely opposed to this change, but I do not completely understand the need for this.

The object_id column in the term relationships table should already use bigint(20 as its data type, making casts unnecessary as far as I am aware. Is this somehow different on your server?

[superdioz]

Hello @arnaudbroes,

at least in MariaDB, for some incomprehensible reason, subqueries ignore the column type and treat the values as if they were of an unknown type. This means that each row/value is parsed generically. This issue, combined with the single-core processing (another odd feature of MariaDB), places a significant strain on the server.

On sites with a "normal" number of posts, the effect isn't noticeable, but it becomes a problem on large websites. We discovered this on a site with over 80k posts, where this query took over 23 seconds (on an 8-core, 12GB RAM server!).

Forcing the cast in the subquery resolves this behavior. It isn't a query error fix but rather a patch to compensate for the poor management of MariaDB. It is harmless for MySQL, so I hope you can include it in the next version so we don't have to manually patch it with every update.

edit: misspelling

[Arnaud Broes]

Hey @superdioz,

Thank you for providing me with some more details.

I just tried it out and it works just fine so I went ahead and created a PR for it in our development repo. We'll get this out in our 4.7.1 update, which is due in two weeks.

[Arnaud Broes]

@superdioz casting caused unexpected issues with lower MySQL versions during QA testing so I switched to CONVERT(tr.object_id, unsigned) instead. Can you confirm that still fixes these performance issues for you?

[superdioz]

@arnaudbroes we tested it on that huge database and performance-wise it's equivalent. If this improves the compatibility with older db it is even a better solution, thank you!

[Arnaud Broes]

@arnaudbroes we tested it on that huge database and performance-wise it's equivalent. If this improves the compatibility with older db it is even a better solution, thank you!

Perfect, thank you for getting back to me. We'll proceed with this alternative approach then. You can expect the update to go out this Thursday.