MySQL Support

[adminExitium 355]

You are basically describing a ORM in almost every other language. It's been ages since I have heard refer to an ORM as a DAL, because it's just a subset of what an ORM provides.

The ORM of choice for most dotnet projects is EFCore (https://github.com/dotnet/efcore/), but moving to that is quite a significant effort. Probably in the order of months of a single developer's dedicated time & focus.

As an example, Jellyfin has been moving towards it for more than a year now (counting the development time). Testing with the initial merge of the movement to EFCore has been happening since Nov. 2024. And this is all with basically no other significant feature added on the server side in the meanwhile.

I don't think Emby can afford such a long pause for any new functionality, especially when there are a lot more impactful and demanded feature requests still open: https://emby.media/community/index.php?/forum/98-feature-requests/&sortby=forums_topics_reactions.topic_reactions&sortdirection=desc.

Edited August 15, 2025 by adminExitium

[ebr 16169]

It all comes down to what are you really trying to accomplish?  This thread was started under the premise that somehow switching to a different database backend would bring all kinds of improvements.  There are 14+ pages of discussion now on why that isn't really the case.

Switching to a different database just for the sake of using a different (more complicated) data store holds no interest for us.  

[softworkz 5066]

The crucial point that people are missing is the fundamental difference between an in-process database like SQLite and external RDBMS like all those that are being mentioned here. 

SQLite needs to be compared with dBase or the MS Jet Engine (MS Access Databases) or ODBC drivers for local data files. programming against such in-process databases involves using different patterns than against external RDBMS. Just to give one - which is also the most prominent - example: You don't do caching at the application level, because it's almost always cheaper to requery the data from the DB when needed. This is different with external dbs. MS Entity Framework (EFCore) like mentioned above, heavily relies on caching, because it's optimized for external RDBMS. When you use that with an SQLite backend, you typically get worse results than with a MSSQL, Oracle or Postgres backend. 
But we are not using any such middleware. We are using SQLite directly, with custom patched and optimized driver-level components and all of our code is optimized for that - in turn getting better results in terms of performance than with external RDBMS.

I have eplained this in many ways before, you just need to go through this conversation (just focus on my posts, you can safely ignore everrything else).

The original poster of this 

7 hours ago, adminExitium said:

As an example, Jellyfin has been moving towards it for more than a year now (counting the development time)

In fact, it has been a major declared goal of theirs, since the very first moment after they "forked". Now it's 8 years later and they still don't have it working properly.

I think there couldn't be any better proof for what we've been stating all over the time in this conversation being valid and justified. 
It's absolutely not impossible to do that, and it would take us just a few months - not 8 years - to get it done, but there are so many other - much more valuable - things we can do in the same amount of time instead, that this whole endeavour simply doesn#t weigh out well enough.

[bakes82 167]

There a much better optimized media servers out there that offer micro services for transcoding multiple nodes for scale. Adding Reddis for caching, and having your flavor of sql for persistent storage. They are all newer frameworks started from ground up. They just don’t have native clients for apps, mainly focused currently on web and getting all the stuff working. Someone just needs to build an agnostic media player with defined schema/api. Kind of like infuse but even they seem to do custom implementations per server. At that point you could write some proxies to conform much like how litellm and many of the ai proxies all conform to openai format.

[softworkz 5066]

On 8/16/2025 at 12:18 AM, bakes82 said:

that offer micro services for transcoding multiple nodes for scale

That's just a worker-pool but not a micro service architecture.

 

On 8/16/2025 at 12:18 AM, bakes82 said:

multiple nodes for scale.

Most important for Emby is live-transcoding, and a single such transcoding job cannot be scaled by employing multiple nodes.

 

On 8/16/2025 at 12:18 AM, bakes82 said:

Adding Reddis for caching

Well - that's one of the points I tried to explain earlier: With an in-process db like SQLite, you need no caching because re-retrieving the data from the in-process database is (overall) much cheaper than using a caching server.

(also, it's Redis, not Reddis)

 

On 8/16/2025 at 12:18 AM, bakes82 said:

They just don’t have native clients for apps, mainly focused currently on web and getting all the stuff working. Someone just needs to build an agnostic media player with defined schema/api. Kind of like infuse but even they seem to do custom implementations per server. At that point you could write some proxies to conform much like how litellm and many of the ai proxies all conform to openai format.

"Just don't have", "someone just needs", "you  could write",...

 

That's a lot of hot steam, mixed up with a few buzz-words, but essentially pointless in total.

Edited August 17, 2025 by softworkz

[TeamB 2438]

5 hours ago, bakes82 said:

There a much better optimized media servers out there that offer micro services for transcoding multiple nodes for scale. Adding Reddis for caching, and having your flavor of sql for persistent storage. They are all newer frameworks started from ground up. 

this sounds cool, do you have any project names?

[Q-Droid 989]

10 hours ago, softworkz said:

"Just don't have", "someone just needs", "you  could write",...

That's a lot of hot steam, mixed up with a few buzz-words, but essentially pointless in total.

You have summarized this entire thread with those few words.

 

[Dibbes 514]

@softworkzat his finest... you gave me a good chuckle here

Edited August 17, 2025 by Dibbes

[shocker 135]

To be honest in the last two years Emby performance improved a lot! There are a little of fine-tuning behind and based on what has been described by @softworkzif you go to external db, in terms of performance without the fine tunning you will not have a good gain. But in case that the performance is not what users seek, external db will provide more room to add custom tweaks, personal improvements, migration etc.

With my personal experience the only poor experience that I currently have is the search. It improved a lot, but I do have cases when results are over 10s and is getting timed out on Lg/Samsung devices. I don’t believe that is a much room of improvement for large libraries, but I have tested JF with Melisearch, and everything is instant.

Also I think there are folks that are waiting for a common db with multiple nodes for load balancing or high availability.

Just my personal opinion.

Cheers!

[softworkz 5066]

On 8/17/2025 at 7:15 PM, shocker said:

To be honest in the last two years Emby performance improved a lot! There are a little of fine-tuning behind and based on what has been described by @softworkzif you go to external db, in terms of performance without the fine tunning you will not have a good gain. But in case that the performance is not what users seek, external db will provide more room to add custom tweaks, personal improvements, migration etc.

Sure!

Like I said before in this conversation - I love the idea of multi-node Emby-Server architectures and had thought out plans for this more than once over the years. 

But you can't always just do what you'd love to do. So let me set this straight once again: If somebody would come along, asking for such kinds of features and willing to pay for the development, then we could implement and deliver these features of course for that customer. It's surely doable - it just requires a substantial amount of work.

Our actual users on the other side - are paying for a "Personal Media Server" and have no need for professional/enterprise features of that kind. It is neither fair nor acceptable to let them pay for the development of features which are far outside their patterns of use and of interest for less than one permille (0.1%)  of users at best.

From a business perspective, it can be expressed in a simple way: Either you have "thousands" paying a few dollars each - or you have a one or a few who are paying thousands of dollars each. Either case, you owe to those who are driving your business, and in this case, it's pretty clear who that is and so we are trying to serve them as best as we can by focusing on features with the highest demands and the greatest benefits for the majority of our users.

[adminExitium 355]

2 hours ago, softworkz said:

pay for the development

Just curious, what kind of figures are you thinking of for this?

[ebr 16169]

1 hour ago, adminExitium said:

Just curious, what kind of figures are you thinking of for this?

Hi.  Its really not in the picture because we cannot afford to pull our resources away from the mainline development even if someone were to pay for it.

[softworkz 5066]

1 hour ago, adminExitium said:

Just curious, what kind of figures are you thinking of for this?

Just to answer the question: It's months of continuous work, preceded by a couple of weeks alone for prototyping, testing out options and working out a solid concept. 

Again - for the short-attention-span readers: this is not about a simple data layer change - it would require fundamental changes at varrious core elements of Emby Server, even down to client apps and APIs in various aspects to get equal results like now with SQLite.

And like @ebrsaid: it cannot go in a way that we'd stop working on things for the Emby user base - which is and has been driving Emby for so many years already.

[Dibbes 514]

1 hour ago, adminExitium said:

Just curious, what kind of figures are you thinking of for this?

It really comes down to how much someone is willing to invest. If you can bill external resources with margin to handle the heavy lifting, then in theory almost anything is possible. In practice, I doubt many would actually pay for it as we're looking at 7 figures.

What often gets overlooked is the overhead that comes with expanding core features. An architectural change like this essentially doubles everything: bugs, testing, release cycles and so on.

Without a clear enterprise licence that companies are willing to pay for at a substantially higher rate than a personal licence, I do not think it is feasible. Personally, I would love to see this feature, but I doubt it would deliver the value people expect in a non-corporate setup.

Edited August 19, 2025 by Dibbes

[softworkz 5066]

27 minutes ago, Dibbes said:

What often gets overlooked is the overhead that comes with expanding core features. An architectural change like this essentially doubles everything: bugs, testing, release cycles and so on.

This hits the point exactly. At a (much) earlier stage this would have been easier to go for, but by now, the debt is so high, that it's very hard to even identify all the cases for which tests would need to be coded, to make sure that the behavior of every little nit remains identical (a parallel mode would probably be required which executes all db operations here and there and validates that results are identical).

Another problem is that many API endpoints (by which client apps are interfacing) are using InternalItemsQuery, and this allows queries for anytrhing about anytrhing with anytrhing filtered by anytrhing or anytrhing else or anytrhing else witrh anytrhing or without anything, inluding anytrhing grouped by anything, sorted by anytrhing, etc. etc.

While this provides great value and flexibility to clients, it is a nightmare for the server-side, because you can optimize queries, retrieval, caching and other processing only for something specifc but not for everything at the same time.
It also allows to specifcy only certain fields to retrieve, which is nice, but it doesn't go together with caching for example - each cached entity needs to have all fields, so you need to decide what to do in which cases...etc.

(just a tiny bite for the taste)

Edited August 19, 2025 by softworkz

[Dibbes 514]

And before people say, this can be simply adjusted or avoided in the apps by building in a query as to which backend is used, the answer to that is: no, not simply.

The technical debt is high. To preserve behaviour you would need a parallel mode with dual writes and read-compare to validate that every operation returns identical results. The test matrix would quite literally explode.

So, while in theory, Emby could support both SQLite and a heavier engine such as MySQL or MSSQL, but that is not the same as just flipping a switch.

The difficulty is ensuring identical behaviour across two very different systems. SQLite is embedded and predictable, while MySQL and MSSQL bring client/server semantics, different planners, transaction models, collation rules, and indexing strategies.

With InternalItemsQuery allowing almost any combination of filters, sorts, groupings, and field-level selections, the test surface becomes massive. Optimisations that work on one engine may need a completely different approach on another. Caching also becomes problematic because partial field selection conflicts with storing complete entities, and now that logic would need to be maintained for both providers.

When it can be simple(-ish):

Read-only, scoped endpoints: Add a /v2 set of endpoints that serve a few hot use cases only. Back them with a separate read model (denormalised tables or materialised views).

One-way sync: Keep the legacy DB as the source of truth and use change data capture or event/outbox to feed the new read model.

No free-form queries: Drop InternalItemsQuery flexibility on v2; expose presets/profiles with fixed filters, sorts, and field sets.

Independent caching: Cache whole entities or fixed projections only, with ETags. No field-level partials.

That lets you run two (or more) kinds of endpoints side by side with limited blast radius.

When it won’t be simple:

Write parity required: The moment v2 must handle writes, you need dual-writes, idempotency, reconciliation, and consistency checks. Complexity jumps.

Full InternalItemsQuery support: Replicating arbitrary filtering/grouping/sorting on a second store recreates the same optimisation and caching problems twice.

So, while supporting both is possible, it means maintaining schemas and migrations for two (or more) engines, validating results across both, and dealing with their differences in search, JSON support, upserts, and isolation levels. Unless the scope is narrowed to something like read-only endpoints or predefined query shapes, these becomes permanent parallel systems and a significant ongoing engineering burden.

In other words, the Emby team won't have the bandwidth to handle this and still have time to handle features that would make a lot more sense for a private media server...

Again, I'd love to have this feature, but I get why not... (also, I prefer to have the ebook feature built out and the TV Next released, but that's just personal preference )

Edited August 19, 2025 by Dibbes

[chowbok 132]

On 8/19/2025 at 4:51 PM, Dibbes said:

The test matrix would quite literally explode.

Pretty sure this isn't true.

[BensProjects 0]

It would be so good to have MySQL support, especially when users want to do load balancing. Locking in SQLite is a big problem here.

Then I could also implement Docker Swarm correctly with HA.

[softworkz 5066]

51 minutes ago, BensProjects said:

MySQL support, especially when users want to do load balancing.Locking in SQLite is a big problem here.

LOL - sure, let's do load-balancing where locks need to be synchronized over the network. What a phenomenal plan!

 

[nunuxx 7]

Jellyfin seems to open the road to another type of database.

Their last 10.11 release going this way.

Quote

We're finalizing our efforts of migrating to an ORM in Jellyfin 10.11 and are working support for different databases in future releases.

https://features.jellyfin.org/posts/315/mysql-server-back-end

[softworkz 5066]

21 minutes ago, nunuxx said:

Jellyfin seems to open the road to another type of database.

Their last 10.11 release going this way.

https://features.jellyfin.org/posts/315/mysql-server-back-end

They are "opening this road" for almost 10 years by now.
It's been one of their primary goals but still not delivered. 
Like I had said many times in this conversation: It's not an easy task - at least when you want to achieve comparable performance. I'll be really glad when they finally have it, because it will eventually vaporize this eternal conversation.

I can already make the following 

Predictions

1. Their performance with different DB backends

• SQLite => Slowest
• ...
• MySQL => Faster
• ...
• MS SQL => Fastest

Very Important: this has nothing to do with performance of the databaases. It's a matter of the ORM framework and the database providers, and the combination MSSQL Server >> MSSQL DB provider + MSSQL EF provider >> MS Entity Framework has seen amounts of investment in development, where the others couldn't even dream of.
 

2. Performance compared to Emby

• Emby vs. JF+SQLite => Emby always better
• Emby vs. JF+MSSQL (similar for MySQL)
  • MediaItems < 500k => Emby always better
  • MediaItems > 500k => Cannot predict, depends on too many factors

Note: For a fair comparison it would be required to compare apples to apples. Emby has a much wider feature set and those more complex db queries, so care needs to be taken to compare equivalent operations.

Let's see how right or wrong I it will turn out to be...
(I won't edit this post, so if there's a typo, then be it)

[nunuxx 7]

@softworkzDon't take it personally, but this thread has been open for nearly 10 years now too, and nothing has changed.

For my needs, I'm encountering some issues with Emby.

After some research, I found that SQLite might be the problem.

I've already tuned what I could with the help of AI, but it's still not enough.

If Jellyfin starts to perform better, I will definitely consider migrating.

It's not about you or anyone else; it's just about having a system that works reliably all the time.

[chowbok 132]

5 hours ago, nunuxx said:

thread has been open for nearly 10 years now too, and nothing has changed.

And for ten years they've been telling you this isn't going to be implemented. Your complaint would make sense if it were one of the many things that has Luke saying "It's definitely possible for the future" every year or so, but in this case they've been pretty clear that this one isn't on the agenda. So the length of time people have been bitching about it is irrelevant. It just indicates that the team is tolerant of whining. If this were a Github issue it would have been closed years ago.

[shocker 135]

To be honest since the request of mySQL there were tons of improvements and changes into the Emby server. Maybe by that time it was a need due to poor performance with large libraries, but nowadays the software is way more faster.

I think we the users need to better to explain why we need mySQL as it might be easier to develop something in addition that changing the entire DB. This seems to be a massive effort and it's not something that it will happen in the near future. There are lot of things to do on Emby where most of us will benefit and the dev time invested on doing this change just for "a feature" might not worth it.

I can see a lot of people saying that they want to have high availability, well if this is the main focus just create two machines, use a virtual ip (keepalived) and sync/move the docker as you wish. If a few minutes downtime is not acceptable for this, then Emby might not be the product you need  

Personal needs:

1. While the search improved a lot with the latest versions (v4.9.1.35 as of today), I still have the results in average of 5 seconds. Sometimes is going up to >10 sec and the results time out on WebOS/TizenOS (rare cases now with the latest beta that handle this even better). This is not a matter of resource, I can keep the entire DB in my RAM memory, but I do have a 3.2GB library and 192GB of RAM. Average RAM usage is 30GB out of 192GB and server load average is <0.2 (linux system). Everything loads instant except the search. JellyFin implemented meilisearch and it's working great, all the results are in milliseconds. I think a similar approach will help a lot.

2. API for the Backup & Restore plugin. It will be beneficial to play with, do some scripts, if you want to sync/migrate a user away to a new docker instance. 

3. Jellyseerr integration within the apps will be a nice touch. Similar on how Streamyfin manage this on JellyFin.

4. Remote/external server for handling transcoding/conversions on user offline sync (downloaded media).

5. Remote servers for transcoding. For example I would like to use a server with few (for balancing the tasks) GPU's only for 4K transcoding. While the main Emby server can focus on managing the metadata and streaming to the users, the hardcore tasks to be moved away and the output of the transcoding session to be read over a file-share mount (this is somehow related with #4).

6. Sub-accounts. Under a user to be able to use the "sub-account functionality". Eg. I will create the user John (master account) with the limit of 1 max simultaneous video stream and then I would like to add two sub-accounts: wife and kid. The limits should persists from the master account, but they should be quickly switch the profile (similar to other streaming platforms of playstation, etc). Switching the user is not very useful if you have more users on the server as you need to search for your name in a big list. I usually hide the users on the server.

7. Manage app/user settings for all your users. Easier to enforce politics then writing to all of your friends on WhatsApp to change something . For example if they have unstable speed for internet connectivity at the peak hours 18:00-23:00, their throughput might fluctuate and Emby will downgrade the streams to 3-4Mbps and trigger the transcoding. For this if they change to maximum, everything is fine, even no buffering as the internet speed can keep up with the streaming needs. It will be nice to be able to enforce this option to a user or all users. Somehow similar on how Streamyfin manage this on JellyFin, but more advanced if this can implemented on server/client side.

8. LiveTV - the promised features that we are all waiting for.

 

This is just my feedback, maybe non of those will be considered into the feature.

Thank you for the massive changes on the performance and keep up the good work! I really love Emby!

[softworkz 5066]

2 hours ago, chowbok said:

7 hours ago, nunuxx said:

thread has been open for nearly 10 years now too, and nothing has changed.

And for ten years they've been telling you this isn't going to be implemented. Your complaint would make sense if it were one of the many things that has Luke saying "It's definitely possible for the future" every year or so, but in this case they've been pretty clear that this one isn't on the agenda. So the length of time people have been bitching about it is irrelevant. It just indicates that the team is tolerant of whining

Thanks @chowbok that's pretty much the answer that I would have given  

Saying "it's possible for the future" is a shortcut for silencing discussion, but it also creates expectations followed by disappointment - which I don't like to be creating. It's not wrong in a way that it will never happen - it's in fact "possible" that under certain circumstances such a change might happen - then probably for very different reasons than those that have been brought up by users here, but as long as it's an unlikely outcome in the short-term, I find it unfair to create wrong expectations, so I  rather try to patiently explain why it's not to be expected any time in the short term, even though - as everybody can see - that's a much harder way than making promises.