Search triggers image lookup requests for all results

[Colin Sullender]

Currently, the search API calls get_artist_image(), get_track_image(), and get_album_image() on ALL results matching the query despite only returning the top 5 results for display on the website. On instances with many scrobbles, this will trigger an enormous number of external image fetches, bringing the server to a crawl and needlessly consuming storage. Consider only parsing the number of responses requested by the search.

Steps to Reproduce

0. Have instance with existing scrobble history
1. Enter the letter "z" into website search bar
2. Server will proceed to lookup images for every artist, album, and track containing the letter "z".

[Casper]

I noticed this issue as well and made my own changes to drastically improve the speed. There's more changes that can be done on the frontend as well such as cancelling outdated requests since it's search-as-you-type but I haven't bothered with that.

Here's my patch, you can apply it with git apply 0001-Fix-search-performance.patch or git am 0001-Fix-search-performance.patch

https://github.com/krateng/maloja/files/14927066/0001-Fix-search-performance.patch

View without downloading

From fb88fa3d29639956b082ae64dc47dcec94eab32a Mon Sep 17 00:00:00 2001
From: Casper S <[e-mail]>
Date: Wed, 10 Apr 2024 05:26:30 +0200
Subject: [PATCH] Fix search performance

---
 maloja/apis/native_v1.py    |  8 ++++----
 maloja/database/__init__.py | 20 ++++++++++----------
 maloja/database/sqldb.py    | 25 +++++++++++++++++--------
 3 files changed, 31 insertions(+), 22 deletions(-)

diff --git a/maloja/apis/native_v1.py b/maloja/apis/native_v1.py
index 1ea2716..6182e89 100644
--- a/maloja/apis/native_v1.py
+++ b/maloja/apis/native_v1.py
@@ -728,13 +728,13 @@ def rebuild(**keys):
 def search(**keys):
 	"""Internal Use Only"""
 	query = keys.get("query")
-	max_ = keys.get("max")
+	max_ = keys.get("max", None)
 	if max_ is not None: max_ = int(max_)
 	query = query.lower()
 
-	artists = database.db_search(query,type="ARTIST")
-	tracks = database.db_search(query,type="TRACK")
-	albums = database.db_search(query,type="ALBUM")
+	artists = database.db_search(query, type="ARTIST", max=max_)
+	tracks = database.db_search(query, type="TRACK", max=max_)
+	albums = database.db_search(query, type="ALBUM", max=max_)
 
 
 
diff --git a/maloja/database/__init__.py b/maloja/database/__init__.py
index 37c159a..8427e96 100644
--- a/maloja/database/__init__.py
+++ b/maloja/database/__init__.py
@@ -963,13 +963,13 @@ def start_db():
 
 
 # Search for strings
-def db_search(query,type=None):
-	results = []
-	if type=="ARTIST":
-		results = sqldb.search_artist(query)
-	if type=="TRACK":
-		results = sqldb.search_track(query)
-	if type=="ALBUM":
-		results = sqldb.search_album(query)
-
-	return results
+def db_search(query, type=None, max=None):
+    results = []
+    if type == "ARTIST":
+        results = sqldb.search_artist(query, max)
+    elif type == "TRACK":
+        results = sqldb.search_track(query, max)
+    elif type == "ALBUM":
+        results = sqldb.search_album(query, max)
+
+    return results
diff --git a/maloja/database/sqldb.py b/maloja/database/sqldb.py
index a51c3e7..fe15c10 100644
--- a/maloja/database/sqldb.py
+++ b/maloja/database/sqldb.py
@@ -1666,30 +1666,39 @@ def get_scrobble(timestamp, include_internal=False, dbconn=None):
 
 @cached_wrapper
 @connection_provider
-def search_artist(searchterm,dbconn=None):
-	op = DB['artists'].select().where(
-		DB['artists'].c.name_normalized.ilike(normalize_name(f"%{searchterm}%"))
-	)
-	result = dbconn.execute(op).all()
+def search_artist(searchterm, max=None, dbconn=None):
+    op = DB['artists'].select().where(
+        DB['artists'].c.name_normalized.ilike(normalize_name(f"%{searchterm}%"))
+    )
+    if max is not None:
+        op = op.limit(max)
+        
+    result = dbconn.execute(op).all()
 
-	return [get_artist(row.id,dbconn=dbconn) for row in result]
+    return [get_artist(row.id, dbconn=dbconn) for row in result]
 
 @cached_wrapper
 @connection_provider
-def search_track(searchterm,dbconn=None):
+def search_track(searchterm, max=None, dbconn=None):
 	op = DB['tracks'].select().where(
 		DB['tracks'].c.title_normalized.ilike(normalize_name(f"%{searchterm}%"))
 	)
+	if max is not None:
+		op = op.limit(max)
+		
 	result = dbconn.execute(op).all()
 
 	return [get_track(row.id,dbconn=dbconn) for row in result]
 
 @cached_wrapper
 @connection_provider
-def search_album(searchterm,dbconn=None):
+def search_album(searchterm, max=None, dbconn=None):
 	op = DB['albums'].select().where(
 		DB['albums'].c.albtitle_normalized.ilike(normalize_name(f"%{searchterm}%"))
 	)
+	if max is not None:
+		op = op.limit(max)
+	
 	result = dbconn.execute(op).all()
 
 	return [get_album(row.id,dbconn=dbconn) for row in result]
-- 
2.34.1

[Colin Sullender]

Very nice! Unfortunately I'm using Docker so I'm not sure how to apply the patch (or have it persist between updates).

[Casper]

Well, you'd probably have to clone the repo, apply the patch and build your own image with docker build -t maloja . -f Containerfile and use that instead of the official one.

Obviously not ideal and it would be nice if @krateng made the change and released a new version instead but there's been no activity for 2 months