Huge amount of reads for `within`. Is this normal?
mmmoli opened this issue Β· comments
Issue
Worried about ongoing costs π
Observation
12k READ queries over a few hours
Expected
Maybe an order of magnitude fewer.
Fixes
Either reset expectation and prepare to pay or fix crazy code?
Context
I've developed a graphql
API to list 27k Projects
stored in Firestore. I'm using geoFirex
to find Projects within a certain radius.
As part of development, I ran this function 30 times or so:
export const getProjectsNearPoint = async (
point: Location,
searchRadiusKm: number = 10,
queryFn: QueryFn = ref => ref.limit(3) // Bug: This is not working
) => {
const center = geo.point(point.latitude, point.longitude);
const field = "location";
const query = geo
.collection(COLLECTION, queryFn)
.within(center, searchRadiusKm, field);
const nearbyProjects = await get(query);
const result = nearbyProjects.map(
({ queryMetadata, ...project }: GeoQueryDocument) => ({
distance: queryMetadata.distance,
...project
})
);
return result;
};
Questions
- Is this normal?
- Would an index help?
- does anyone suspect my graphql resolvers are doing anything necessarily recursive?
Thanks.
Did you find any fix to this ?. I also tested the code and when I receive 4 points on my map I have 800 reads in firebase.
Additional reads beyond the radius result is expected, but a bug was fixed in 0.1.0 where get
was not unsubscribing from the underlying callback - I suspect that was the root of your issue.
Limit does not work as you might expect. A regular firestore query is made for 9 different square hashes, so your worst case limit would actually be applied as 9 x 3. I don't recommend using limit, but rather a smaller query radius or where
with ==
.
Also, you can now enable logging to track the total documents returned in a query within(c, r, f, { log: true })
to monitor the results clientside.
@mmmoli I am looking to using this library but I am curious to know if this issue is resolved for you as of this moment as per update suggested by @codediodeio ?
To be honest, I freaked out.
Between that and spending $30k in 72 hours I decided it wasn't worth it and moved away.
I think that's more of reflection of Fb's pricing model that this library though π
@VincentDufour I didn't. I wasn't part of that engineering team.
From my link to the blog post describing their disaster:
This means that every session to our site read the same number of documents as we have of number of payments. #UnaVacaPorDeLaCalle received more than 16,000 supporters, so: 2 million sessions x 16,000 documents = more than 40 Billion requests to Firestore on less than 48 hours.
In summary: think, hard.