High number search queries on off

Question

High number search queries on off

raphael0202 opened this issue 24 days ago · comments

Raphaël Bournhonesque commented 24 days ago

I found out in the logs that Smoothie app may send a high amount of search queries in a short timeframe:

IPADDR - - [11/Jun/2024:11:47:41 +0000] "POST /cgi/search.pl HTTP/1.0" 200 72 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/28075
IPADDR - - [11/Jun/2024:11:47:41 +0000] "POST /cgi/search.pl HTTP/1.0" 200 72 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/25631
IPADDR - - [11/Jun/2024:11:47:41 +0000] "POST /cgi/search.pl HTTP/1.0" 200 72 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/32324
IPADDR - - [11/Jun/2024:11:47:45 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/6523
IPADDR - - [11/Jun/2024:11:47:45 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7089
IPADDR - - [11/Jun/2024:11:47:45 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/19965
IPADDR - - [11/Jun/2024:11:47:46 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/6032
IPADDR - - [11/Jun/2024:11:47:46 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7255
IPADDR - - [11/Jun/2024:11:47:46 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7036
IPADDR - - [11/Jun/2024:11:47:41 +0000] "POST /cgi/search.pl HTTP/1.0" 200 72 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 6/6415146
IPADDR - - [11/Jun/2024:11:47:48 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7321
IPADDR - - [11/Jun/2024:11:47:48 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7236
IPADDR - - [11/Jun/2024:11:47:48 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7397
IPADDR - - [11/Jun/2024:11:47:48 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/20328
IPADDR - - [11/Jun/2024:11:47:50 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/6618
IPADDR - - [11/Jun/2024:11:47:50 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7064
IPADDR - - [11/Jun/2024:11:47:50 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/6844
IPADDR - - [11/Jun/2024:11:47:50 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/20934
IPADDR - - [11/Jun/2024:11:47:45 +0000] "POST /cgi/search.pl HTTP/1.0" 200 72 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 6/6779922
IPADDR - - [11/Jun/2024:11:47:46 +0000] "POST /cgi/search.pl HTTP/1.0" 200 72 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 6/6724017
IPADDR - - [11/Jun/2024:11:47:58 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7077
IPADDR - - [11/Jun/2024:11:47:58 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/7933
IPADDR - - [11/Jun/2024:11:47:58 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/6421
IPADDR - - [11/Jun/2024:11:47:58 +0000] "POST /cgi/search.pl HTTP/1.0" 200 74 "-" "- Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - https://world.openfoodfacts.org/ -  - Smoothie - OpenFoodFacts - 4.13.1+1279 - android+SP1A.210812.016.G975FXXSGHWC1 - 68700e1f-aa28-49dc-b300-d34cc387bad8" 0/6695

There is probably a bug underneath.

Pierre Slamich · Answer 1 · Tue Jun 11 2024 21:53:51 GMT+0800 (China Standard Time)

Hypothesis: Hidden in dev mode, we have the offline mode, which could be doing such queries, but we have never advertised it
@g123k

monsieurtanuki · Answer 2 · Sun Jun 16 2024 16:07:16 GMT+0800 (China Standard Time)

Just had this issue: when you go to the "welcome monsieurtanuki" page, we automatically compute the number of products you added, you modified, you photographed, you need to complete:

each time you go to this page, it's triggered
each time those are four search.pl queries in parallel

<!DOCTYPE html><html><head><meta name="robots" content="noindex"></head><body>
<h1>TOO MANY REQUESTS</h1>
<p>You are sending too many requests to our servers.</p><p>To know more about the rate limits we enforce, please refer to the <a href="https://openfoodfacts.github.io/openfoodfacts-server/api/#rate-limits">rate-limit section in our documentation</a>.</p>
<p>If you need to download data about a large number of products, it's preferable to <a href="https://world.openfoodfacts.org/data">download a data dump</a>. If this is unexpected, contact us on Slack or write us an email at <a href="mailto:contact@openfoodfacts.org">contact@openfoodfacts.org</a>.</p>
</body></html>

Edouard Marquez · Answer 3 · Mon Jun 17 2024 01:16:30 GMT+0800 (China Standard Time)

We have a project to redesign this tab and I think we should do it, even without the real concept of "For me".
And in this particular case, it would make sense to cache data and even give some range (e.g. : instead of 7, let's say between 5 and 10), which would be OK until the next refresh

Pierre Slamich · Answer 4 · Mon Jun 17 2024 21:27:07 GMT+0800 (China Standard Time)

@raphael0202 could you put the search parameters ? It would help us pinpoint which part of the app is faulty.
We could also add more instrumentation Smoothie side.

Edouard Marquez · Answer 5 · Mon Jun 17 2024 21:45:32 GMT+0800 (China Standard Time)

FYI We can track all network requests with the Flutter DevTools.
But there is nothing suspicious on the search and the profile.
It might be related to poor network conditions.

Raphaël Bournhonesque · Answer 6 · Tue Jun 18 2024 19:55:59 GMT+0800 (China Standard Time)

@raphael0202 could you put the search parameters ? It would help us pinpoint which part of the app is faulty.

No it's a POST request, we don't have access from nginx logs to the content