Jump to content

MySQL Support


dcrdev
Abobader
Message added by Abobader,

Last warning: Attacking other members & team will cause a ban for your account.

Recommended Posts

yeah it'd be simply not letting you continue if you cant provide external database information on wizard, so you can either hit back and not choose advanced database setup and choose the easy button or figure out wtf an external db server is, why you'd want one and if its worth the setup..

We need no UI for editing DB config as thats often required to be done offline w/out app running, give us a database.conf with examples commented out, and a .sql file with the expected structure, we dont even need a wizard tbh but I know the devs dislike idea of having config files to edit from other requests I've made. If they ever implement this it'll have a wizard I suspect.

Dont have to support converting backends after the fact, almost nothing else does. The most we could ask for here is an easy api for dumping the internal db into a standard .sql file ready import, but the community can tool up scripts to dump sqlite to a database server, then edit the configs by hand for the of us willing to travel down such a path massaging our old data into a new database.. that right there would block any pleb not able to run a database server from ever trying it out just because it was there, yet the power users would not be hindered.

Edited by nayr
  • Like 1
Link to comment
Share on other sites

Dibbes

@nayr I'd even go as far to get this without a possibility to dump SQL lite into a real database and set everything up from scratch... it would cost me a week of scanning or so and my users would have to do there play-states manually again, but IMO it would be worth it...

Link to comment
Share on other sites

ayup, I'd be a bit sad but totally willing to give up years of data to perform this migration if thats what it took to maintain supportability in the future. 

  • Like 1
Link to comment
Share on other sites

kingy444
1 hour ago, Dibbes said:

@nayr I'd even go as far to get this without a possibility to dump SQL lite into a real database and set everything up from scratch... it would cost me a week of scanning or so and my users would have to do there play-states manually again, but IMO it would be worth it...

Play state can be restored via backup plugin 😉

  • Like 1
Link to comment
Share on other sites

Dibbes
23 hours ago, kingy444 said:

Play state can be restored via backup plugin 😉

and exactly where is the playstate stored?

Link to comment
Share on other sites

kingy444
49 minutes ago, Dibbes said:

and exactly where is the playstate stored?

Not sure exactly the location in the table schema it is - but look at the backup plugin - it allows you to backup playstates and restore them selectively - even mapping playstates to different users if you want

i took the curl calls from manual backup restore and automated to keep second server somewhat in sync (only writing one way)

Link to comment
Share on other sites

Dibbes
3 minutes ago, kingy444 said:

Not sure exactly the location in the table schema it is - but look at the backup plugin - it allows you to backup playstates and restore them selectively - even mapping playstates to different users if you want

i took the curl calls from manual backup restore and automated to keep second server somewhat in sync (only writing one way)

So you mean in the SQLite DB...??

Link to comment
Share on other sites

kingy444
48 minutes ago, Dibbes said:

So you mean in the SQLite DB...??

Na mate just go to plugins and find the Server Configuration Backup plugin - it does the backup for you - believe the playstates are exported in XML but not home to confirm

 

1751146C-4091-4AC4-93AC-E7F876901754.jpeg

Link to comment
Share on other sites

  • 2 months later...
dcol

I also would love to see MySQL for Emby. Most of my issues with Emby are database related and I find it very difficult to work with SQLite and the utilities are far superior with MySQL.

  • Like 1
Link to comment
Share on other sites

11 minutes ago, dcol said:

I also would love to see MySQL for Emby. Most of my issues with Emby are database related and I find it very difficult to work with SQLite and the utilities are far superior with MySQL.

Do you know SQLiteStudio? It's at least better than the other one (DB Browser or so)

https://sqlitestudio.pl

Link to comment
Share on other sites

dcol

I really never touch it since it is so 'touchy'. I have to make a new one right now becase of constant SQLite errors in the logs.

Link to comment
Share on other sites

  • 2 months later...
gnufreak

I've started looking for a thread on MySQL support for Emby recently and found this.

I use Emby via a docker image. Emby /config and /data reside on NFS4.1 mounted filesystems. That's how I share data between physical machines and VMs. It works without issues for most software. What doesn't work, is using SQLite over NFS (locking issues). It's nothing new.

Before I started using Emby, I checked its requirements - there was no stated requirement its data needs to be on a local storage. And indeed, it used to work without many issues in the past.

I don't use it every day, or every week and I don't want to babysit it. However, most times when having guests, who'd I like to give access to Emby to, it becomes a total embarrassment. That's entirely due to SQLite DB corruption. I've tried recovering the DB after stopping Emby. Great, works again. Breaks again after starting library scan a few minutes later.

Today - fresh install (again). Cleanly configured, everything's fine. Adding a first library which resides over NFS. No problem yet.

Still watching logs, I click "Scan library files". Keeps chugging along, then:

$ grep SQLiteExceptio ../logs/embyserver.txt

       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)

[...] - hundreds more exceptions of this sort there.


I remember seeing a post from one of the devs (can't find it now), which to paraphrase said:

"MySQL support is NOT a priority since sqlite works fine".

It does not: fresh install. Corruption within minutes. Not just likely but guaranteed, given a very moderate library size.

I've paid for lifetime Emby Premiere, based on the requirements and the fact that it used to work not perfectly but acceptably enough. That's not the case anymore.

Could a real relational database support be prioritized or clearly stated "no, we won't do it?", in which case I'd like to request a partial refund and move on to something else? It doesn't need to be the default but it cannot be a "bastard child" feature either with a disclaimer "you're using this on your own risk".

Thanks

Edited by gnufreak
Link to comment
Share on other sites

2 hours ago, gnufreak said:

I've started looking for a thread on MySQL support for Emby recently and found this.

I use Emby via a docker image. Emby /config and /data reside on NFS4.1 mounted filesystems. That's how I share data between physical machines and VMs. It works without issues for most software. What doesn't work, is using SQLite over NFS (locking issues). It's nothing new.

Before I started using Emby, I checked its requirements - there was no stated requirement its data needs to be on a local storage. And indeed, it used to work without many issues in the past.

I don't use it every day, or every week and I don't want to babysit it. However, most times when having guests, who'd I like to give access to Emby to, it becomes a total embarrassment. That's entirely due to SQLite DB corruption. I've tried recovering the DB after stopping Emby. Great, works again. Breaks again after starting library scan a few minutes later.

Today - fresh install (again). Cleanly configured, everything's fine. Adding a first library which resides over NFS. No problem yet.

Still watching logs, I click "Scan library files". Keeps chugging along, then:

$ grep SQLiteExceptio ../logs/embyserver.txt

       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)

[...] - hundreds more exceptions of this sort there.


I remember seeing a post from one of the devs (can't find it now), which to paraphrase said:

"MySQL support is NOT a priority since sqlite works fine".

It does not: fresh install. Corruption within minutes. Not just likely but guaranteed, given a very moderate library size.

I've paid for lifetime Emby Premiere, based on the requirements and the fact that it used to work not perfectly but acceptably enough. That's not the case anymore.

Could a real relational database support be prioritized or clearly stated "no, we won't do it?", in which case I'd like to request a partial refund and move on to something else? It doesn't need to be the default but it cannot be a "bastard child" feature either with a disclaimer "you're using this on your own risk".

Thanks

It's something we're interested in but we prioritize media management and consumption features first.

Link to comment
Share on other sites

4 hours ago, gnufreak said:

I've started looking for a thread on MySQL support for Emby recently and found this.

I use Emby via a docker image. Emby /config and /data reside on NFS4.1 mounted filesystems. That's how I share data between physical machines and VMs. It works without issues for most software. What doesn't work, is using SQLite over NFS (locking issues). It's nothing new.

Before I started using Emby, I checked its requirements - there was no stated requirement its data needs to be on a local storage. And indeed, it used to work without many issues in the past.

I don't use it every day, or every week and I don't want to babysit it. However, most times when having guests, who'd I like to give access to Emby to, it becomes a total embarrassment. That's entirely due to SQLite DB corruption. I've tried recovering the DB after stopping Emby. Great, works again. Breaks again after starting library scan a few minutes later.

Today - fresh install (again). Cleanly configured, everything's fine. Adding a first library which resides over NFS. No problem yet.

Still watching logs, I click "Scan library files". Keeps chugging along, then:

$ grep SQLiteExceptio ../logs/embyserver.txt

       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
       SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
          at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)

[...] - hundreds more exceptions of this sort there.


I remember seeing a post from one of the devs (can't find it now), which to paraphrase said:

"MySQL support is NOT a priority since sqlite works fine".

It does not: fresh install. Corruption within minutes. Not just likely but guaranteed, given a very moderate library size.

I've paid for lifetime Emby Premiere, based on the requirements and the fact that it used to work not perfectly but acceptably enough. That's not the case anymore.

Could a real relational database support be prioritized or clearly stated "no, we won't do it?", in which case I'd like to request a partial refund and move on to something else? It doesn't need to be the default but it cannot be a "bastard child" feature either with a disclaimer "you're using this on your own risk".

Thanks

Sorry for your woes. The devs have already admitted (shamefully) that they've de-prioritized adding this "feature", and that if they did they would monetize it for corporations to use since they've recently realized the benefits of non-local databases. Yes, we all know this is lame, but not all teams can be proponents of modern development practices. Some older folks need help understanding micro-services or highly available systems. Either that or they just need to realize people will eventually use something else that does keep up with the times. 

I hear Jellyfin supports external DBs it and has active development again. Fingers crossed we work it out one way or another soon.

Edited by rsvg
Link to comment
Share on other sites

gnufreak
3 hours ago, Luke said:

It's something we're interested in but we prioritize media management and consumption features first.

Luke,

I've tried to be as diplomatic as possible, given how poor and limiting that backend data storage design decision was.
Taking into account that this feature request is over 5 years old now, the statement "we're interested in it" doesn't bear any value here.
Based on that, I don't trust your team has any intention or capability of ever implementing this or prioritizing features / bug fixes with the benefit of your users in mind.

If the published requirements stated your software required its data to be stored on a locally attached storage, or it would otherwise be unusable, I'd have looked elsewhere.
Instead, I've paid the price and was hit by this bug rendering your software entirely unusable later.

In real world it's called a manufacturing defect. In software development, it'a s critical, show-stopping bug. Not a feature request.

How can I receive a partial refund?

Thanks

Edited by gnufreak
  • Confused 1
Link to comment
Share on other sites

@gnufreak I'm afraid, but you are mixing two things that are completely unrelated. You are trying to make up a relation between a bug in the application and a design decision.

2 hours ago, gnufreak said:

If the published requirements stated your software required its data to be stored on a locally attached storage,

I can't follow that argument. It would be rather the opposite:
If the application would require a RDBS for operation, then you could expect this to be stated in the application requirements.

When you are experiencing a bug, I would like to kindly ask you to report this in a new topic and we'll do our best to resolve it; should the issue persist, you can surely ask to be refunded, but please give us a chance to look into it.
Please also see how to report a problem. Thanks !

Link to comment
Share on other sites

3 hours ago, rsvg said:

The devs have already admitted (shamefully) that they've de-prioritized adding this "feature", and that if they did they would monetize it for corporations to use since they've recently realized the benefits of non-local databases

Everybody can read through the history of this thread to see that this is a weird distortion from what has been said. What you are referring to was in the context of adding the ability to run a cluster of multiple Emby Server nodes implementing load-balancing and failover capabilities as a scenario where a non-local database would make sense or even be a requirement.

Link to comment
Share on other sites

roaku
3 hours ago, gnufreak said:

Luke,

I've tried to be as diplomatic as possible, given how poor and limiting that backend data storage design decision was.
Taking into account that this feature request is over 5 years old now, the statement "we're interested in it" doesn't bear any value here.
Based on that, I don't trust your team has any intention or capability of ever implementing this or prioritizing features / bug fixes with the benefit of your users in mind.

If the published requirements stated your software required its data to be stored on a locally attached storage, or it would otherwise be unusable, I'd have looked elsewhere.
Instead, I've paid the price and was hit by this bug rendering your software entirely unusable later.

In real world it's called a manufacturing defect. In software development, it'a s critical, show-stopping bug. Not a feature request.

How can I receive a partial refund?

Thanks

The 'published requirements' don't guarantee Emby will use a database at all, much less one you can operate independently of the app itself.

  • Like 2
Link to comment
Share on other sites

gnufreak
15 hours ago, softworkz said:

@gnufreak I'm afraid, but you are mixing two things that are completely unrelated. You are trying to make up a relation between a bug in the application and a design decision.

I'm not "mixing, trying or making anything up". Focusing on understanding the point I've made instead of trying to discredit it first would have been beneficial here.

The bug, which manifests itself in SQLite DB getting corrupted, namely:

SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed

renders a freshly installed instance of Emby unusable when it attempts a library scan, relates directly to the fact that it uses SQLite.
I'm not going to go into details of why your DB becomes corrupted, as the relevant ones are (or rather *should* have been) known by any developer considering using it:

https://www.sqlite.org/howtocorrupt.html

 

To address your remark where you misunderstood and failed to follow the argument:

I use NFS to share data between applications running on my VM instances. That's a pretty standard practice.

  1. I checked out Emby - I liked its interface and the fact that it wasn't a PITA to set up behind a reverse proxy.
  2. I decided I wanted to use it.
  3. I read its requirements.
  4. No show stoppers were described.
  5. I didn't care for the requirements to mention "it doesn't use a real RDBMS" (which roaku@ thought was relevant for some reason) or that it DOES use a RDBMS.
  6. What I did care about but wasn't mentioned: we use SQLite, which means you cannot store the data on filesystem that is NOT locally attached and expect it to work.

SQLite on NFS does not work, unless your code is perfect and perfectly controls concurrency and does not rely on locking to do that for you. That's a common knowledge, or rather: should be, at least among those familiar with DBs, which should be the people making those architectural decisions. Workarounds are just that. They never work in the long term. SQLite + NFS = no go, sooner or later (hint: sooner). Simple as that.

Had this been mentioned: "Emby is unable to store its data on non-local FS, if you do this, you're on your own", I'd have looked elsewhere.

To make it easier to visualize: when you buy a car, you buy a product. You don't take the engine apart to see how it's been put together to find potential points of failures. When there are issues with it, only then you open it up to see what caused them.

And then shake your head, seeing that the issue is a consequence of using parts unfit for purpose.

That's what this is and people are telling you this, yet you keep dismissing it and prefer debugging stuff which are guaranteed to break vs using the right tool for the job and calling the show-stopper a "feature" with a low priority of fixing.

Edited by gnufreak
Link to comment
Share on other sites

I'm afraid, but your story appears to be constructed. On one side, you claim to know very well about problems of file databases on network storage and on the other side, you are complaining that you haven't been warned about it. Installing and running applications on network storage involves many caveats, not only with regards to file databases. Somebody who is used to work with such setups should know that and would not blindly expect an application to work in that way.
Emby doesn't state anywhere that such setups can be expected to work and that's sufficient from my point of view. Listing scenarios that will not work is somewhat pointless anyway, as that would result in an almost infinite list. 

Edited by softworkz
Link to comment
Share on other sites

pir8radio

I mean softworkz has a point....   

you say:

What doesn't work, is using SQLite over NFS (locking issues). It's nothing new.

Before I started using Emby, I checked its requirements - there was no stated requirement its data needs to be on a local storage. And indeed, it used to work without many issues in the past.

sqlite over nfs doesn't work and its "nothing new"   yet you checked the requirements and had to see emby uses sqlite, and already knowing you cant use it on nfs installs, still seem surprised that its "requirements don't mention that data needs to be on local storage".    In your buying a car example that's like looking at the sticker for the car, buying it, then bringing it back saying it doesn't fly....  But the sticker doesn't say anything about this car not being able to turn into an airplane and fly...   No duh... why would you think that? we never said the car could fly...   Just like emby never claimed the DB can be ran over NFS.    Has someone done that? maybe...   can someone make a car fly, sure..  But that's outside of the scope of how either the car or emby was originally designed.  

I want a beefier DB like most people on this thread, but the argument you are trying to make isn't valid, emby is made to be installed on a single machine and run for home users.   when you do the install for emby server, is there anywhere in the setup that says "would you like to install the DB on a network drive?"?   NO, because that's not how its meant to be used.  It's a want,  I want it to, but we wait until more than a 100 or so hard-core users want the feature..  the other 98% want other things like new live tv features etc.. Those things are higher in priority due to the majority wanting/needing them.    I'm sure they will look into a more robust DB soon enough..   and if not.......

 

Edited by pir8radio
  • Like 3
Link to comment
Share on other sites

koopa8840
20 hours ago, pir8radio said:

I mean softworkz has a point....   

you say:


What doesn't work, is using SQLite over NFS (locking issues). It's nothing new.

Before I started using Emby, I checked its requirements - there was no stated requirement its data needs to be on a local storage. And indeed, it used to work without many issues in the past.

sqlite over nfs doesn't work and its "nothing new"   yet you checked the requirements and had to see emby uses sqlite, and already knowing you cant use it on nfs installs, still seem surprised that its "requirements don't mention that data needs to be on local storage".    In your buying a car example that's like looking at the sticker for the car, buying it, then bringing it back saying it doesn't fly....  But the sticker doesn't say anything about this car not being able to turn into an airplane and fly...   No duh... why would you think that? we never said the car could fly...   Just like emby never claimed the DB can be ran over NFS.    Has someone done that? maybe...   can someone make a car fly, sure..  But that's outside of the scope of how either the car or emby was originally designed.  

I want a beefier DB like most people on this thread, but the argument you are trying to make isn't valid, emby is made to be installed on a single machine and run for home users.   when you do the install for emby server, is there anywhere in the setup that says "would you like to install the DB on a network drive?"?   NO, because that's not how its meant to be used.  It's a want,  I want it to, but we wait until more than a 100 or so hard-core users want the feature..  the other 98% want other things like new live tv features etc.. Those things are higher in priority due to the majority wanting/needing them.    I'm sure they will look into a more robust DB soon enough..   and if not.......

 

Funny how you completely overlooked, this statement "What I did care about but wasn't mentioned: we use SQLite, which means you cannot store the data on filesystem that is NOT locally attached and expect it to work." No where is it mentioned, without having to dig for the info, that Emby Server uses SQLite. gnufreak even stated if he had known that information before downloading, he wouldn't have downloaded Emby Server. I guess you have selective reading since you were more focused on his car example and giving your wild and inaccurate example. Emby Server implies it can be ran from an NFS due to its lack of documentation, but whatever. It has become quite clear certain people prefer to argue and "be right", than admit Emby actually has flaws; most being pushed by the developers. 

Link to comment
Share on other sites

On 7/31/2021 at 10:15 PM, pir8radio said:

 when you do the install for emby server, is there anywhere in the setup that says "would you like to install the DB on a network drive?"?   NO, because that's not how its meant to be used.

It wouldn't say "would you like to install the DB on a network drive", that's asinine. During the EXISTING setup wizard, it would say "default: Local DB" or "Enter your database path and credentials". Emby doesn't have a basic setup option that Wordpress v1 had in 2004 lol. 

Link to comment
Share on other sites

roaku
36 minutes ago, rsvg said:

It wouldn't say "would you like to install the DB on a network drive", that's asinine. During the EXISTING setup wizard, it would say "default: Local DB" or "Enter your database path and credentials". Emby doesn't have a basic setup option that Wordpress v1 had in 2004 lol. 

There's reply confusion because someone is complaining about something unrelated to the main discussion topic.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...