Giters

doug-martin / nestjs-query

Easy CRUD for GraphQL.

Home Page:https://doug-martin.github.io/nestjs-query

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

__nestjsQuery__entityIndex__ reduces performance of my database view

thehappycoder opened this issue · comments

commented

I don't have code to reproduce the issue without sharing the entire project but I can explain it.

I've got a view ListingApplicationChannelStats that is optimised to be selected by listingApplicationChannelId but nestjs-query is joining it in a way that Postgres can't use the index anymore. I wish it was possible to add indexes on view columns like id but it's not possible.

I think nestjs-query is joining by id here due to an assumption that id is a primary key, and, therefore, it's indexed:

SELECT "listingApplicationChannelStats"."id" AS "listingApplicationChannelStats_id",
       "listingApplicationChannelStats"."listingApplicationChannelId" AS "listingApplicationChannelStats_listingApplicationChannelId",
       "listingApplicationChannelStats"."listingId" AS "listingApplicationChannelStats_listingId",
       "listingApplicationChannelStats"."messageAuthorId" AS "listingApplicationChannelStats_messageAuthorId",
       "listingApplicationChannelStats"."messageId" AS "listingApplicationChannelStats_messageId",
       "listingApplicationChannelStats"."memberId" AS "listingApplicationChannelStats_memberId",
       "listingApplicationChannelStats"."hasRead" AS "listingApplicationChannelStats_hasRead",
       "listingApplicationChannelStats"."readAt" AS "listingApplicationChannelStats_readAt",
       "unioned"."__nestjsQuery__entityIndex__" AS "__nestjsQuery__entityIndex__"
FROM "ListingApplicationChannelStats" "listingApplicationChannelStats"
INNER JOIN
  (SELECT *
   FROM
     (SELECT "listingApplicationChannelStats"."id" AS "listingApplicationChannelStats_id",
             0 AS "__nestjsQuery__entityIndex__"
      FROM "ListingApplicationChannelStats" "listingApplicationChannelStats"
      WHERE ("listingApplicationChannelStats"."listingApplicationChannelId" = 'fdf93cb1-5e77-4f17-b309-17c125f74754')
        AND ("listingApplicationChannelStats"."memberId" = '5ed3c5f6-5874-4e61-83f2-993650dc7ecb')) AS "listingApplicationChannelStats"
   UNION ALL SELECT *
   FROM
     (SELECT "listingApplicationChannelStats"."id" AS "listingApplicationChannelStats_id",
             1 AS "__nestjsQuery__entityIndex__"
      FROM "ListingApplicationChannelStats" "listingApplicationChannelStats"
      WHERE ("listingApplicationChannelStats"."listingApplicationChannelId" = 'd7329db8-bafa-4322-bd82-5fc124fa3198')
        AND ("listingApplicationChannelStats"."memberId" = '5ed3c5f6-5874-4e61-83f2-993650dc7ecb')) AS "listingApplicationChannelStats"
   UNION ALL SELECT *
   FROM
     (SELECT "listingApplicationChannelStats"."id" AS "listingApplicationChannelStats_id",
             2 AS "__nestjsQuery__entityIndex__"
      FROM "ListingApplicationChannelStats" "listingApplicationChannelStats"
      WHERE ("listingApplicationChannelStats"."listingApplicationChannelId" = 'd15cc379-69c3-4445-8522-7a9202384a50')
        AND ("listingApplicationChannelStats"."memberId" = '5ed3c5f6-5874-4e61-83f2-993650dc7ecb')) AS "listingApplicationChannelStats"
   UNION ALL SELECT *
   FROM
     (SELECT "listingApplicationChannelStats"."id" AS "listingApplicationChannelStats_id",
             3 AS "__nestjsQuery__entityIndex__"
      FROM "ListingApplicationChannelStats" "listingApplicationChannelStats"
      WHERE ("listingApplicationChannelStats"."listingApplicationChannelId" = '696fe371-3f43-4cad-bf82-912cb6aa6ef8')
        AND ("listingApplicationChannelStats"."memberId" = '5ed3c5f6-5874-4e61-83f2-993650dc7ecb')) AS "listingApplicationChannelStats"
   UNION ALL SELECT *
   FROM
     (SELECT "listingApplicationChannelStats"."id" AS "listingApplicationChannelStats_id",
             4 AS "__nestjsQuery__entityIndex__"
      FROM "ListingApplicationChannelStats" "listingApplicationChannelStats"
      WHERE ("listingApplicationChannelStats"."listingApplicationChannelId" = 'b2903302-dc4d-4175-a3a3-83eabd66557e')
        AND ("listingApplicationChannelStats"."memberId" = '5ed3c5f6-5874-4e61-83f2-993650dc7ecb')) AS "listingApplicationChannelStats") "unioned" ON "listingApplicationChannelStats"."id" = "unioned"."listingApplicationChannelStats_id"
ORDER BY "unioned"."__nestjsQuery__entityIndex__" ASC

The view:

SELECT
	"ListingApplicationChannels".id || '_' || "ListingApplicationChannelMembers"."memberId" AS id,
	"ListingApplicationChannels".id AS "listingApplicationChannelId",
	"ListingApplicationChannels"."listingId",
	"LatestMessages"."authorId" AS "messageAuthorId",
	"LatestMessages"."messageId",
	"ListingApplicationChannelMembers"."memberId",
	("MessageReceipts"."readByUserId" IS NOT NULL OR "ListingApplicationChannelMembers"."memberId" = "LatestMessages"."authorId") AS "hasRead",
	"MessageReceipts"."createdAt" AS "readAt"
FROM
	"ListingApplicationChannels"
JOIN
	"ListingApplicationChannelMembers" ON "ListingApplicationChannelMembers"."listingApplicationChannelId" = "ListingApplicationChannels".id
LEFT JOIN (
	SELECT
		"Messages"."listingApplicationChannelId",
		(array_agg("Messages"."authorId" ORDER BY "Messages"."createdAt" DESC))[1] AS "authorId",
		MAX("Messages"."createdAt") AS "sentAt",
		(array_agg("Messages".id ORDER BY "Messages"."createdAt" DESC))[1] AS "messageId"
	FROM
		"Messages"
	WHERE
		"Messages".category = 'REGULAR'
	GROUP BY
		"Messages"."listingApplicationChannelId"
) "LatestMessages" ON (
		"LatestMessages"."listingApplicationChannelId" = "ListingApplicationChannels".id
	)
LEFT JOIN
	"MessageReceipts" ON (
		"LatestMessages"."messageId" = "MessageReceipts"."messageId" AND ("MessageReceipts"."readByUserId" = "ListingApplicationChannelMembers"."memberId")
	)
ORDER BY
	"ListingApplicationChannels".id,
	"LatestMessages"."sentAt" DESC

Is there some workaround for this?

commented

Rubber 🦆 !!!

Seems to be performant after changing the primary key from id to the composite one in ListingApplicationChannelStat view entity:

@PrimaryColumn()
  listingApplicationChannelId: string

  @PrimaryColumn()
  memberId: string
commented

It's still an issue though as it stops using an index when there is some number of union-ed selections. I think it will be faster to run multiple queries instead of batching them like that.

If it's possible to disable this batch query for this particular relation, that would be awesome!

commented

I think I had a similar issue but only that the performance was bad when it needed to fetch a lot of relations, I fixed this in my fork with this commit. I rewrote the service to use AND WHERE "primaryKey" IN (... array of keys) instead of joins.

If you are also using TypeORM you could also checkout that.