Account linking queries do a sequential scan on the users/identities table by default in managed Supabase

[M]

Bug report

• I confirm this is a bug with Supabase, not with my own application.
• I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

I'm currently working on a project with >1 million users in managed Supabase and we were recently seeing heavy database load as a large number of new users were signing up. Looking at the query performance report data, we found two queries taking up 80% of our compute:

• SELECT identities.created_at, identities.email, identities.id, identities.identity_data, identities.last_sign_in_at, identities.provider, identities.provider_id, identities.updated_at, identities.user_id FROM identities AS identities WHERE email ilike any ($1)
• SELECT users.aud, users.banned_until, users.confirmation_sent_at, users.confirmation_token, users.confirmed_at, users.created_at, users.deleted_at, users.email, users.email_change, users.email_change_confirm_status, users.email_change_sent_at, users.email_change_token_current, users.email_change_token_new, users.email_confirmed_at, users.encrypted_password, users.id, users.instance_id, users.invited_at, users.is_sso_user, users.last_sign_in_at, users.phone, users.phone_change, users.phone_change_sent_at, users.phone_change_token, users.phone_confirmed_at, users.raw_app_meta_data, users.raw_user_meta_data, users.reauthentication_sent_at, users.reauthentication_token, users.recovery_sent_at, users.recovery_token, users.role, users.updated_at FROM users AS users WHERE email ilike any ($1) and is_sso_user is false

Running those queries manually indicated that the referenced tables were being sequentially scanned by these queries and the indexes were not being used. I believe I was able to trace the calls done to the account linking code, see:

https://github.com/supabase/gotrue/blob/5ad703bddc6ec74f076cbe6ce1f942663343d47a/internal/models/linking.go#L109-L122

So, I'm creating this bug report here, but if I'm wrong about this, please move this to where it does belong.

To Reproduce

1. Have a lot of users
2. Create a new user using OAuth
3. Watch the requests take a while to complete

Expected behavior

Account creation should be quick even when there are many users.

[Kang Ming]

@tasgon thanks for catching this! we've made a fix in #1394 and it should help speed things up quite abit once it's merged and rolled out

[Alexander Cui]

When will this be rolled out? I'm still seeing very degraded performance

[Joel Lee]

@alexcdot the fix will likely be rolled out within a week and a half - could we trouble you to open a ticket if you need it sooner? Where possible we'll do our best to expedite a fix

Thanks

Update: I see the ticket, we'll reply there - thanks!