Database joins not using indexes
mvdbeek opened this issue · comments
Here's an example of an innocent looking join:
galaxy_main=> EXPLAIN ANALYZE select count(hda.id) from history_dataset_association as hda join history on hda.history_id = history.id where history.user_id = 14950;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----
Finalize Aggregate (cost=269291.58..269291.59 rows=1 width=8) (actual time=4027.339..4058.823 rows=1 loops=1)
-> Gather (cost=269291.16..269291.57 rows=4 width=8) (actual time=4014.864..4058.809 rows=4 loops=1)
Workers Planned: 4
Workers Launched: 3
-> Partial Aggregate (cost=268291.16..268291.17 rows=1 width=8) (actual time=4008.114..4008.116 rows=1 loops=4)
-> Nested Loop (cost=0.57..268290.61 rows=220 width=4) (actual time=238.847..4006.160 rows=14379 loops=4)
-> Parallel Seq Scan on history (cost=0.00..132860.36 rows=14 width=4) (actual time=162.247..543.692 rows=65 loops=4)
Filter: (user_id = 14950)
Rows Removed by Filter: 2084020
-> Index Scan using ix_history_dataset_association_history_id on history_dataset_association hda (cost=0.57..9647.74 rows=2585 width=8) (actual time=8.002..53.229 rows=221 loops=2
60)
Index Cond: (history_id = history.id)
Planning Time: 2.296 ms
Execution Time: 4058.977 ms
Subqueries WITH LIMIT are fine:
galaxy_main=> EXPLAIN ANALYZE select count(hda.id) from history_dataset_association as hda where hda.history_id in (select id from history where history.user_id = '14950' LIMIT 10000);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=567382.68..567382.69 rows=1 width=8) (actual time=237.826..237.829 rows=1 loops=1)
-> Nested Loop (cost=105.16..566993.29 rows=155755 width=4) (actual time=0.579..234.579 rows=57517 loops=1)
-> HashAggregate (cost=104.59..105.19 rows=60 width=4) (actual time=0.564..0.682 rows=260 loops=1)
Group Key: history.id
-> Limit (cost=0.56..103.84 rows=60 width=4) (actual time=0.020..0.488 rows=260 loops=1)
-> Index Scan using ix_history_user_id on history (cost=0.56..103.84 rows=60 width=4) (actual time=0.019..0.468 rows=260 loops=1)
Index Cond: (user_id = 14950)
-> Index Scan using ix_history_dataset_association_history_id on history_dataset_association hda (cost=0.57..9422.18 rows=2596 width=8) (actual time=0.110..0.878 rows=221 loops=260)
Index Cond: (history_id = history.id)
Planning Time: 0.268 ms
Execution Time: 237.872 ms
Altering the cost means the index gets used:
galaxy_main=> set random_page_cost = 1.2;
SET
galaxy_main=> EXPLAIN ANALYZE select hda.id from history_dataset_association as hda inner join history on hda.history_id = history.id where history.user_id = 1;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------
Nested Loop (cost=1.13..173803.72 rows=889 width=4) (actual time=23.184..426.394 rows=201 loops=1)
-> Index Scan using ix_history_user_id on history (cost=0.56..32.78 rows=61 width=4) (actual time=9.963..68.653
rows=22 loops=1)
Index Cond: (user_id = 1)
-> Index Scan using ix_history_dataset_association_history_id on history_dataset_association hda (cost=0.57..282
3.07 rows=2563 width=8) (actual time=5.206..16.253 rows=9 loops=22)
Index Cond: (history_id = history.id)
Planning Time: 5.453 ms
Execution Time: 426.584 ms
(7 rows)
galaxy_main=>
Potentially linked issues are that we were creating a lot of new histories recently due to bugs in the anon history handling.
Somewhat related, galaxyproject/galaxy#17725 is about removing userless and contentless histories.
It's not just joins. Here's a simple count from a tmp table. On test, we only have:
galaxy_test=> select count(*) from tmp_unused_history;
count
--------
314629
And when we check the job table against this one, we get an index scan:
galaxy_test=> explain select count(*) from job where history_id in (select id from tmp_unused_history);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=81753.19..81753.20 rows=1 width=8)
-> Merge Join (cost=0.85..78129.70 rows=1449399 width=0)
Merge Cond: (job.history_id = tmp_unused_history.id)
-> Index Only Scan using ix_job_history_id on job (cost=0.43..50844.97 rows=1449979 width=4)
-> Index Only Scan using tmp_unused_history_pkey on tmp_unused_history (cost=0.42..8277.75 rows=355215 width=4)
On main, we have x16 that amount:
galaxy_main=> select count(*) from tmp_unused_history;
count
---------
5087218
But we get a sequential scan:
galaxy_main=> explain select count(*) from job where history_id in (select id from tmp_unused_history);
QUERY PLAN
-----------------------------------------------------------------------------------------------------------
Aggregate (cost=5481131.94..5481131.95 rows=1 width=8)
-> Hash Join (cost=174084.69..5349960.43 rows=52468604 width=0)
Hash Cond: (job.history_id = tmp_unused_history.id)
-> Index Only Scan using ix_job_history_id on job (cost=0.56..4605792.53 rows=52470353 width=4)
-> Hash (cost=79910.50..79910.50 rows=5740050 width=4)
-> Seq Scan on tmp_unused_history (cost=0.00..79910.50 rows=5740050 width=4)
Aren't you returning the majority of rows from tmp_unused_history? Then a seq scan makes sense ?
select id returns all rows, but it's an index. Shouldn't it use an index scan, as it's doing for test?
That's up to the planner, but I think that's a legitimate choice ... if a large fraction of the table is to be returned an index scan provides no advantage and the involved random seek may push the planner to a seq scan.
This explains it better: https://www.pgmustard.com/blog/why-isnt-postgres-using-my-index
random_page_cost has been dropped from 4.0 to 1.2 on Main today
Applied by hand to galaxy-db-02 last week and then permanently to the new DB server in galaxyproject/infrastructure-playbook@63cc270#diff-5b735970294a4d340921576497eaf6bab9a5feb327afce62319c06b2ca9f35d0R189