Search Very Slow (3 minute response)

[kingom 14]

On 7/26/2026 at 7:17 PM, Luke said:

The browser also has added cancellation of the previous search when you add another letter, and now this cancellation makes it all the way up to the database query. This helps a lot.

 But not all apps can do that and some of the ones that can haven’t gotten a store update with it yet:

In the past, search was working with the LG App.
Problems started when updating the Server to 4.9.1.90

[Rumzzz 74]

New beta dropped with search improvements!! Will vigorously test soon 

 

[bandit8623 251]

22 minutes ago, Rumzzz said:

New beta dropped with search improvements!! Will vigorously test soon 

 

vigorous searches!!

[Chillout 131]

With .21 beta still experiencing long searches that make the server unresponsive.

searching: star wars    - results are immediate

searching for: war  -spinning circle for minutes causing server to lag

I dont mind searches that take long, I don't  like how one user can cause problems for everyone else.

 

 

Edited July 30 by Chillout

[Rumzzz 74]

@Chilloutare you on 4.10.0.21?

I'm also still running into server locking searches (both single and multi word) for a handful of my tests, although it seems less often than before. 

Certain searches that block the server will eventually bring back results if I wait long enough or try enough times. Once results return for a problematic search phrase, subsequent searches for the same phrase seen to return more reliably (cached?).

I think we have 2 different issues people are reporting here:

1) Search queries aren't efficient / take too long

2) Querying the database causes all sessions to lock up

Edited July 30 by Rumzzz

[Luke 42951]

How does 4.10.0.22 compare?

[Chillout 131]

I dont notice much change.... .some 22 searches still lock up the server for other users.

searching for:  black mirror  -near instant results

searching for: black   -locks things up for  ~60sec until it shows results

 

[Rumzzz 74]

I appreciate the recent work on this.

In my initial testing of 22 I do feel a moderate improvement, but still a handful of searches (mostly small/common words) will timeout, and will hold up the server while searching (although the timeout limit seems to be reduced, leading to less time blocking the server for others).

I do like the recently searched section though! Not sure when that was added, but it does seem to have a few issues I can post in a different thread. The loading of this list alone takes almost 15 seconds for me, so that kinda seems like it might add to the timeouts if a search happens before this list loads.

I'm still continuing to test and collect feedback from my other users.

Edited August 2 by Rumzzz

[Napsterbater 42]

Web UI seems ok on .22, from Android App still seems to lock up the server for others.

[blinky 6]

I ended up installing the stable 4.9.50 version and searching works as expected now, no lockups at all so I'll be sticking to it for now

 

Also my hardware is 

AMD ryzen 5 5600x with 16gb ram running windows 11 pro, and about 84tb of storage which most of the drives are full, I need more storage soon so expensive atm

[crash1015 9]

On 8/2/2026 at 4:35 PM, blinky said:

I ended up installing the stable 4.9.50 version and searching works as expected now, no lockups at all so I'll be sticking to it for now

 

Also my hardware is 

AMD ryzen 5 5600x with 16gb ram running windows 11 pro, and about 84tb of storage which most of the drives are full, I need more storage soon so expensive atm

So does this mean I need to reinstall? I'm using 4.9.50 and I'm still having issues. I just don't want to reinstall unless it's certainly going to fix it. 

[Rumzzz 74]

Looking at the release notes on git, looks like 4.9.1.80 was the introduction of the fuzzy-search changes (Improve results of multi-word searches), so I'm assuming any version previous to this one will not have the issue.

https://github.com/MediaBrowser/Emby.Releases/releases#release-4.9.1.80

[crash1015 9]

14 hours ago, Rumzzz said:

Looking at the release notes on git, looks like 4.9.1.80 was the introduction of the fuzzy-search changes (Improve results of multi-word searches), so I'm assuming any version previous to this one will not have the issue.

https://github.com/MediaBrowser/Emby.Releases/releases#release-4.9.1.80

Downgrading from what i have now sounds like a daunting task. Won't the database have issues?

[Rumzzz 74]

1 hour ago, crash1015 said:

Downgrading from what i have now sounds like a daunting task. Won't the database have issues?

Yes, which is exactly what's preventing myself from downgrading, because the database changes typically require a clean install. 

You can try a dirty install of an older version (after you've backup up your system folder), but do so at your own risk with no guarantees.

[Rumzzz 74]

New beta 4.10.0.23 seems promising! Did a quick handful of searches and so far they all came back within ~15 seconds, some near instantly!

I've got much more in-depth testing I will do later, but would love to hear from others.

Edited August 9 by Rumzzz

[Rumzzz 74]

Ok so after some more testing I did run into a few queries that timed out, again mostly smaller/more common words, but it's MUCH less often than before. The small amount of timed out queries I ran into eventually worked if I retried them.

Not sure if the recently searched list is affecting this, but it does take a while for that list to load, and will sometimes time out or will load after my search query.

Still seems like the Emby process doesn't use as many threads/cores as is available on my 13600k when searching. The server does still seem to ramp up my fans and  lock up during longer searches, though the amount of time it's locked has lessened noticably.

I know how hard it can be fixing efficiencies when different users report different things, so thank you for the continued work on this. Looking forward to hearing from other users running on different hardware/platforms.

[Napsterbater 42]

4.10.0.23 still has issues from Android clients for me, same symptoms as before.

[Chillout 131]

.23 beta seems to have fixed the slow search for safari, web, and windows app so I am happy.  TY devs.

 

 

[Napsterbater 42]

On 8/10/2026 at 4:15 AM, Napsterbater said:

4.10.0.23 still has issues from Android clients for me, same symptoms as before.

The same exact search results in WebUI work, yet word for word in the Android App locks up the server for minutes.

[Rumzzz 74]

I think the search is massively improved however there are still occasionally a few search queries that timeout.

I think an important part of this problem is how the server locks up during search. Even with the vast majority of successful searches, it's still 15-20 seconds of server downtime anytime a user does a search (I can't even get the dashboard to load).

Is that something we can have looked into? Should that merit its own discussion thread?

[Napsterbater 42]

9 minutes ago, Rumzzz said:

I think the search is massively improved however there are still occasionally a few search queries that timeout.

I think an important part of this problem is how the server locks up during search. Even with the vast majority of successful searches, it's still 15-20 seconds of server downtime anytime a user does a search (I can't even get the dashboard to load).

Is that something we can have looked into? Should that merit its own discussion thread?

In my experience with the Android app searching, it is far longer than 15-20 seconds, it can be up to like 3 minutes +/-, however, as I said before, same search terms work perfectly fine on web UI, and I don't use search a lot especially from the web UI because I'm mainly using Android clients, So I don't know if there's some magic combination that would cause it to happen on the web UI as well, and if it does happen, is it the same 3 minutes, or is it shorter.

 

[ruedi 6]

Have the same issue here since a while - it is not only on searching with explicit names.

Just have yout CD collection and try from master directory a random play.

Causes my PC the wait a long time - sometimes it will playx, sometimes not.

Database needs a speed up - on bigger amount of entries it will downgrade in speed now.

A suggestion would be a kind of vaccumisation (as with postgres) or switching to mariaDB - both can handle a bigger amount of data.

But this is a design decision ....

regards,

Rüdiger

[ruedi 6]

29 minutes ago, ruedi said:

Have the same issue here since a while - it is not only on searching with explicit names.

Just have yout CD collection and try from master directory a random play.

Causes my PC the wait a long time - sometimes it will playx, sometimes not.

Database needs a speed up - on bigger amount of entries it will downgrade in speed now.

A suggestion would be a kind of vaccumisation (as with postgres) or switching to mariaDB - both can handle a bigger amount of data.

But this is a design decision ....

regards,

Rüdiger

Let me add some things ... 

OS running: Debian 13, completely patched as virtual client under Proxmox

Emby-Version: 4.10.0.27

Saw now a different behavior, potentially we have a second porblem

Using the App under Windows 11: search process to create the file list is running, host = 100% load. After a while CPU load is decreasing to 4% -> Windows App is not playing (circle is turning, Mous operations are possibl, no sound)

 

Using the Web-App with Firefox under Windows 11: search process to create the file list is running, host = 100% load. After a while CPU load is decreasing to 4% -> Sound is playing

 

So it seems, there is a communication problem to the app, to.

 

And ... yes, there is an option to enable "vacuumisation" on the DB, but only on next start of server, which can take months. Possibly we need this to do more often as a subtak - possibly at night. A bit strange.... my library.db is 939474944 Bytes, after restart with vacuumisation it stay the same, but library.db-wal has the same size. I did not expect this ..

regards,

Rüdiger

 

 

[flord22 7]

Got a bit tired of waiting so I let Claude try to do a fix for this and it did it. Search is close to instant on anything I try now. There were 2 issues it fixed. Each issue is explained in its own post one after another here then a guide how to apply it and how to revert it. in case if emby team ever fixes it in their update so you can re run their update afterwards. 

4.10.x: "Recent searches" query (WasSearched/DateLastSearched) freezes entire server for 1–2 minutes on large libraries

Versions affected (observed): 4.10.0.20, 4.10.0.23, 4.10.0.25 (Linux/Ubuntu 24.04, .NET 8.0.28, bundled SQLite 3.53.3). Did not occur on 4.9.x. Library size: ~191,000 MediaItems, ~555,000 UserDatas rows.

Symptom

Opening the search page in Emby Web fires GET /Users/{id}/Items?SortBy=DateLastSearched,SortName&SortOrder=Descending&Limit=20&Recursive=true&EnableTotalRecordCount=false&WasSearched=true (the "recent searches" panel — fired before the user types anything). On our server this single query runs 87–117 seconds of pure CPU inside sqlite3_step() (one thread pinned at 100%, zero disk I/O, verified with gdb + strace). While it runs, effectively the whole server convoys behind it: complex item queries and user-data writes queue (/Sessions/Playing/Progress POSTs took 89–99s; requests observed starved up to 15 minutes across consecutive episodes), then everything flushes in the same second the query completes. To all users the server appears completely hung.

Typed search itself is NOT the culprit on our server: the FTS-based SearchTerm queries measure 27–850ms standalone (they only appeared slow because they queued behind the recent-searches query). This is likely the same class of issue reported in "Search Very Slow (3 minute response)" (topic 144128).

Root cause (verified)

The SQL (recovered verbatim from a process core dump taken mid-freeze; abbreviated):

select ..., (Select ShareLevel from UserItemShares join AncestorIds2
             on AncestorIds2.AncestorId=UserItemShares.ItemId
             where UserItemShares.UserId=1 and UserItemShares.ShareLevel not null
               and AncestorIds2.ItemId=A.Id order by Distance limit 1) as ShareLevel
from mediaitems A
left join UserDatas on A.UserDataKeyId=UserDatas.UserDataKeyId And UserDatas.UserId=1
where A.Type in (1,2,5,6,8,...,34)
  AND UserDatas.DateLastSearchedInt > 0
  AND (ShareLevel > 0 OR A.Type in (...) OR A.IsPublic=1)
  AND A.ExtraType is null
  AND ( EXISTS (SELECT 1 FROM AncestorIds2 WHERE itemid=A.Id AND AncestorId in (...))
     OR EXISTS (ListItems join ancestorids2 ...)
     OR EXISTS (itemPeople2 JOIN AncestorIds2 ...)
     OR EXISTS (ItemLinks2 join ancestorids2 ... Type in (...))
     OR EXISTS (ItemLinks2 ItemLinks2TwoLevel WHERE EXISTS (...) ...) )
Group by A.PresentationUniqueKey
ORDER BY MAX(UserDatas.DateLastSearchedInt) DESC NULLS LAST,
         A.SortName collate NATURALSORT DESC LIMIT 20

There is no index on UserDatas.DateLastSearchedInt, so the planner cannot see that DateLastSearchedInt > 0 is extremely selective (only ~10–100 rows in the whole table ever have it set). SQLite then picks this plan (EXPLAIN QUERY PLAN from a copy of the production DB):

SEARCH A USING INDEX idx_MediaItems47cd2 (type=? AND ExtraType=?)   -- ~176k rows
  CORRELATED SCALAR SUBQUERY 1 (ShareLevel)  -- incl. USE TEMP B-TREE FOR ORDER BY, per row!
  CORRELATED SCALAR SUBQUERIES 2..7 (the visibility EXISTS chain, per row)
SEARCH UserDatas USING INDEX UserDatasIndexUnique1 (UserDataKeyId=? AND userId=?)  -- applied LAST

i.e. it scans ~176k items and evaluates the ShareLevel subquery (with a per-row sort) plus up to five correlated visibility subqueries for every row, and only then joins UserDatas where DateLastSearchedInt > 0 throws away all but ~10 rows. Measured: >90 s. The mirror plan (drive from UserDatas first) takes ~2 s; which plan a pooled connection gets appears to depend on the sqlite_stat1 state at prepare time, which is why the freeze is intermittent and per-connection ("plan roulette" — we observed the identical request take 116,798 ms and, four minutes later, 2,351 ms). ANALYZE does NOT stabilize the good plan (after a fresh ANALYZE the planner chose SCAN A, still >60 s).

Fix that works (verified on production)

CREATE INDEX idx_UserDatas_LastSearched
ON UserDatas(UserId, DateLastSearchedInt)
WHERE DateLastSearchedInt IS NOT NULL;

Build time 0.2 s (partial index over the handful of rows that have the column set). The planner then drives from this index in every stats state we tested. Result on the live server: the recent-searches request went from 116,798 ms → 16 ms, and the server-wide freezes stopped. Suggest shipping this index (or equivalent) in the 4.10 schema, and/or restructuring the query so the selective DateLastSearchedInt predicate drives the join rather than being applied after the per-row ShareLevel/visibility subqueries.

Notes

• No SQLite errors of any kind in logs (no "database is locked", no corruption); PRAGMA quick_check ok. The DB stays fully readable by other processes during the freeze (external probe queries answered in 34–52 ms throughout) — the hang is entirely this query plus the connection-pool convoy behind it.

• Reproduction needs: a multi-user library in the 100k+ item range, at least one user with DateLastSearchedInt set (i.e. someone has used search before), and folder-restricted users (the ShareLevel/visibility subqueries).

• Secondary observation: Debug App: Sqlite: 284 - automatic index on ... warnings appear frequently for LastWatchedEpisodes(SeriesPresentationUniqueKey) and LinkedCounts(AId) (materialized subqueries in the NextUp/Resume queries) — worth a look for the same reason, but not the freeze trigger here.

[flord22 7]

4.10.x: SearchTerm queries take 10–30 s when a term has many matches — degenerate sqlite_stat1 makes per-row visibility subqueries scan whole folder subtrees

Versions observed: 4.10.0.25 beta (also present on 4.10.0.20/.23). Ubuntu 24.04, .NET 8.0.28, bundled SQLite 3.53.3. Library: ~191,000 MediaItems, ~477,000 AncestorIds2 rows, ~555,000 UserDatas rows.

This is a separate issue from my other report about the "recent searches" (WasSearched/DateLastSearched) query freezing the whole server — same server, different query family and different root cause. It is likely the underlying mechanism behind the long-standing "Search Very Slow (3 minute response)" reports (topic 144128), because the cost scales with the number of FTS matches, which is what makes common/multi-word terms pathological.

Symptom

Typed search time scales with the number of FTS matches, at a huge per-match cost — even though the final result is tiny:

SearchTerm	FTS matches	Server time
fellow	18	0.2 s
lord of the	44	12.8 s
lord	272	12.7–13.7 s (measured repeatedly)
the	29,266	~27 s

That is ~50 ms per matched item. Occasionally the identical request completes in 3 ms — pooled connections cache prepared statements, so each connection keeps whatever plan it compiled first ("plan lottery"; see mechanism below). Search fires 2 of these queries per keystroke (/Items + /ItemTypes), so one user typing can occupy multiple pool connections for ~13 s each.

Root cause

The /Items?SearchTerm=... SQL (recovered verbatim from a process core dump) joins fts_search9 MATCH @SearchTerm to MediaItems — that part is fine and fast. But for every matched row it evaluates a per-row visibility OR-chain (abbreviated):

... AND ( EXISTS (SELECT 1 FROM AncestorIds2 WHERE itemid=A.Id AND AncestorId IN (<15 allowed folder ids>))
      OR EXISTS (SELECT 1 FROM ListItems JOIN ancestorids2 ON ListItems.ListItemId=ancestorids2.itemid
                 AND ancestorids2.AncestorId IN (<15 ids>) WHERE ListItems.ListId=A.Id)
      OR EXISTS (SELECT 1 FROM itemPeople2 JOIN AncestorIds2 ... WHERE itemPeople2.PersonId=A.Id ...)
      OR EXISTS (SELECT 1 FROM ItemLinks2 JOIN ancestorids2 ... WHERE ItemLinks2.LinkedId=A.Id ...)
      OR EXISTS (... ItemLinks2 two-level variant ...) )

On this system sqlite_stat1 contained degenerate statistics:

AncestorIds2 | idxAncestorIds2_1 | 470105 1 1     <-- claims 1 row per AncestorId

Reality: 476,645 rows, 146,460 distinct AncestorIds, and an extremely skewed distribution — the top folder has 89,352 descendant rows, and the 15 folder ids in the IN-list cover ~85,000+ rows. Believing "1 row per AncestorId", the planner drives those inner EXISTS joins from ancestorids2 (AncestorId=?) — i.e. it enumerates entire folder subtrees once per matched item (EXPLAIN QUERY PLAN showed SEARCH ancestorids2 USING COVERING INDEX idxAncestorIds2_1 (AncestorId=?) as the outer side of the ListItems/ItemLinks2/ItemPeople2 subqueries). ~55 ms per row × hundreds of matches = 10–30 s per keystroke.

The degenerate stats look like the output of an approximate ANALYZE (PRAGMA optimize with an analysis_limit) sampling a heavily skewed index — the totals (470105 vs current 476645) show they were produced recently on this very database, and a limited sample of AncestorIds2 easily lands in the mostly-unique region and concludes "1 row per key". If the server re-runs limited analysis periodically, good stats get regressively overwritten.

Fix verified on this server

A full, unlimited ANALYZE corrected the estimates (avg 4 rows/AncestorId), which flips the inner join order to the item-side (ListItems (ListId=A.Id), itemPeople2 (PersonId=A.Id), ...):

• lord: 13 s → ~1.0 s end-to-end

• lord of the: 12.8 s → ~1.0 s

• No regressions measured in the NextUp/Resume or other query families.

Important operational detail: ANALYZE does not change the schema cookie, so pooled connections holding cached prepared statements never re-plan and keep the old behavior indefinitely. The running server only picked up the new stats after we forced a schema-cookie bump (create+drop of a dummy index). If Emby runs ANALYZE as maintenance, it should invalidate/re-prepare its cached statements afterwards, or the fix silently doesn't apply until restart.

Remaining structural cost (even with correct stats)

SearchTerm=the (29,266 matches) still takes ~27 s at ~1.1 ms/row, because:

• The query computes count(*) OVER() AS TotalRecordCount even though the client sends EnableTotalRecordCount=false — forcing full enumeration of all matches before LIMIT 50 can apply.

• Each row evaluates a ShareLevel scalar subquery containing ORDER BY Distance LIMIT 1 — a temp-sort per row — plus the visibility EXISTS chain. This expensive per-row form is used for users that have UserItemShares rows; the variant generated for folder-restricted users instead materializes the allowed-items set once as a CTE (WITH WithAncestors AS (...) ... A.Id IN WithAncestors) and measures ~5× cheaper per row.

Suggested fixes:

• Honor EnableTotalRecordCount=false (skip the window count).

• Use the materialized-CTE visibility form for all user types instead of per-row EXISTS chains.

• Ship correct/full statistics (or add explicit indexes/query shapes that don't depend on stat1 being right), and re-prepare cached statements after any ANALYZE.

Where the degenerate stats come from (confirmed) — and why TV clients hit this hardest

Found the mechanism in the server's own configuration defaults: system.xml on this install has <DatabaseAnalysisLimit>5000</DatabaseAnalysisLimit> together with <OptimizeDatabaseOnShutdown>true</OptimizeDatabaseOnShutdown>. Every shutdown therefore runs a limited ANALYZE (analysis_limit=5000) over indexes like AncestorIds2 whose leading column is massively skewed — a 5000-row sample lands in the mostly-unique region and records "1 row per key". So even after a manual full ANALYZE fixes the stats, the next shutdown re-poisons them. Suggested fix: use analysis_limit=0 (full) for these indexes, ship static correct stats, or don't let optimize overwrite better stats.

TV clients make the per-match cost catastrophic in practice: they search per keystroke starting from the first letter. Captured live (Tizen TV app, same behavior on LG): typing "STI…" fired SearchTerm=S (96,295 FTS matches on this library), then SearchTerm=ST (9,118 matches) — the ST pair of queries (/Items + /ItemTypes) each ran 52 s, and with the default MaxLibraryDatabaseConnections=5 the whole server convoyed behind them (playback progress POSTs queued up to 34 s; other users' home screens 30–40 s). One person typing on a TV remote = server-wide outage of ~1 minute, even with corrected stats, because of the O(matches) enumeration described above.

Workaround verified on this server (admin-level, at your own risk)

Because sqlite_stat1 only stores per-index averages (this build has no STAT4), even a correct full ANALYZE (avg 4 rows/AncestorId) cannot express that the specific folder ids used in visibility clauses are the skewed giants — so the planner keeps choosing folder-side scans inside the per-row EXISTS subqueries. Overriding the average for the AncestorId-leading index with a deliberately pessimistic value flips every visibility subquery to the cheap item-side probes:

UPDATE sqlite_stat1 SET stat='476645 3000 1'
WHERE tbl='AncestorIds2' AND idx='idxAncestorIds2_1';
-- then force cached statements to re-plan (ANALYZE/stat changes don't bump the
-- schema cookie): create+drop any dummy index, or restart the server.

Measured results on the live server (191k-item library, ~30 concurrent users):

SearchTerm	before	after
ST (2 letters, 9,118 matches)	52 s	0.08 s
the (29,266 matches)	27 s	0.25 s
lord (272 matches)	13 s	0.008 s

No regressions observed in NextUp/Resume/home-screen query families. Note OptimizeDatabaseOnShutdown/limited ANALYZE will overwrite this row, so the override must be reapplied after any stats maintenance (we disabled OptimizeDatabaseOnShutdown). This is presented as evidence of the root cause and a stopgap — the proper fix belongs in the query/schema (see suggestions above).

Reproduction guidance

Large library (100k+ items) with a deep folder hierarchy (skewed AncestorIds2), a user with UserItemShares rows and a folder allow-list, and degenerate stat1 (run a limited/approximate ANALYZE to get there, or inject '470105 1 1'-style rows). Then search a term with a few hundred matches and compare against a term with <20 matches: the time difference is entirely the per-row visibility evaluation.

No SQLite errors are logged at any point; the database passes PRAGMA quick_check; this is purely query-plan behavior.