ConditionalOffer.products() queries Product children and exludes them again.
gwaidacher opened this issue · comments
Hello!
Issue Summary
I had a problem with requests concerning Offers taking a lot of time:
We have a database of > 3,000.000 products and a simple ConditionalOffer with 40 products without children added and no included classes and categories.
OfferList and OfferDetail request time was over 11 seconds, which is unacceptable.
Digging into this issue, I found out that in Range.product_queryset() Q-joins to collect Products' children are passed into the query:
offer.abstract_models.py, Lines 988 & 997 in AbstractRange.product_queryset():
| Q(parent__includes=self),
(note: this will later be called "children-join")
At last, in ConditionalOffer.products() these children are excluded again, so the included children-joins seem to be unnecessary for me:
offer.abstract_models.py, Line 454 in AbstractConditionalOffer.products():
return queryset.filter(is_discountable=True).exclude(
structure=Product.CHILD)
Caching the products with @cached_property works with a short list of products, but it`s useless if the OfferDetailView is paginated. Getting back from page 2 to page 1 the formerly cached page 1 is requested again with > 11 seconds.
I committed a patch #4042, which reduces the query time significant if there are no classes in the range; if the included products have no children, the remaining LEFT OUTER JOINs are omitted, so the time for gathering the merge joins (~ 8 seconds in the original query) is saved. The remaining INNER JOINS are insignificant in time.
My Tests worked, reducing query time from ~ 11 seconds to far below 1 second.
Speedup factor is > 1000 !
Steps to Reproduce
- Create a Database with 3,000.000 products without children
- Create a simple ConditionalOffer with 40 products
- Query OfferListView and OfferDetailView
Technical details
- Python version: 3.9.2
- Django version: 3.2.16
- Oscar version: 3.2.0
== UPDATE ==
My patch #4046 fixes this Issue by eliminating huge joins when the included products have no classes or no categories or no children
see #4066