Library data disappeared - EmbyServer 4.4.2.0

[Guest]

 

I searched the forum and found nothing, but I was sure that I would find several topics with the same problem.

 

Yesterday, two friends reported that all the data in their bibliographies had disappeared and when checking my Emby, everything seemed normal, so I understood that it would be a problem caused by them and I did not give much importance.

 

It turns out that at the end of yesterday night, already at dawn, some customers reported that they were unable to reproduce any video and that the images had all disappeared.

Upon analyzing, I found that in fact most of the data is gone.

 

So I did a new general screening of the libraries and confirmed that Emby did the process from 0% to 94% in a maximum of one hour of time, but now, eight hours later, the advance was from 94% to 94.6% only, that is, this is absolutely abnormal.

 

 

I thought to send the log files, but I do not know how valid it is, since the last log files sent have not received attention and I still have no answer, as is the case with high memory consumption.

 

I would like to receive some concrete answer for this problem to be solved.

 

Thank you.

 

 

PS: I set up for Emby to save the nfos and arts in the media folders, which are on the google drive.

Edited June 3, 2020 by CarlosLima

[chjohans 34]

The same thing has happened to me recently. I restored my library dB files from a few d a ys previous to the incident and re-scanned my libraries. That got everything up and running again as normal but I've no idea what caused the issue in the first place.

[Guest]

The same thing has happened to me recently. I restored my library dB files from a few d a ys previous to the incident and re-scanned my libraries. That got everything up and running again as normal but I've no idea what caused the issue in the first place.

 

I set up for Emby to save the nfos and arts in the media folders, which are on the google drive.

[chjohans 34]

I save my nfo's and other metadata on a NAS, and I sync play states (played or not played, partially played is not sync'ed) with the trakt plugin, so I guess I could just have deleted the library db files and rebuilt the library without loosing much info. I have done that in the past, but this time I decided to just recover a backup since I have one

[Guest]

Since yesterday, more than 20 hours after the start, tracking has increased from 94% to just 95% and this creates great stress with customers, with numerous complaints.

 

If the arts and nfos are in the media folders, why does Emby need such an absurd time to restore the data? Shouldn't it be easier to have all the data already?

 

And what about backup, what is it for anyway?

 

It is really necessary for developers to find a way to make this more practical and less stressful, especially much faster.

 

If the files are stored in the media folders and if there is a backup, the replacement of all lost data, due to system failure, shouldn't be something resolved in a matter of a few minutes ??

 

With the floor, the developers, please.

 

 

         

Edited June 4, 2020 by CarlosLima

[Happy2Play 9780]

Personally I disable all providers per library in a rebuild process, so only local metadata and images are read, then re-enable them when the initial scan is completed and Refresh as needed.

Edited June 4, 2020 by Happy2Play

[Guest]

Personally I disable all providers per library in a rebuild process, so only local metadata and images are read, then re-enable them when the initial scan is completed and Refresh as needed.

 

 

Thank you for your message.

I use Kodi for my personal media and in it, with the data in the media folders, the trackers will never fetch data from the internet. Doesn't Emby work that way too?

So only now, 5 years after using Emby, did this information reach my knowledge.

So even if there is data (nfo, arts, etc.) in the media folders, will Emby still get it on the internet ??

This seems somewhat inconsistent, since once we have the data, there is no reason for further searches.

I will interrupt the tracking process (which looks like it will be completed in weeks) and I will disable all trackers and redo everything from the beginning.

Thank you

[Happy2Play 9780]

There are still things like people that local metadata will on provide a Name that still has to be searched do to the way Emby handles people now.  But the majority of info can be read from existing nfo files.

 

I recently has my 24 drive card (drivepool) go offline and Emby did its usual clearing all media as the drive no longer existed and it only took the day to re-add everything, but everyones setup is different so time will vary.

[Guest]

Understand.

Well, I interrupted the scanning process, which had been 95% paralyzed in the last 22 hours and followed your tip, turning off the scanners in all libraries.

I didn't really know that Emby searches for metadata on the internet, even though it's all in the media folders. This seems to be something simple to solve, like Kodi, who has a field to mark and he despises all trackers and uses what they call LIO (Local Information Only).

What about Emby losing everything without user action? This is quite frustrating. Can't avoid it?

And lastly, shouldn't the backup, which I do daily, be the solution in critical moments like this?

Thank you

[Happy2Play 9780]

Understand.

 

Well, I interrupted the scanning process, which had been 95% paralyzed in the last 22 hours and followed your tip, turning off the scanners in all libraries.

 

I didn't really know that Emby searches for metadata on the internet, even though it's all in the media folders. This seems to be something simple to solve, like Kodi, who has a field to mark and he despises all trackers and uses what they call LIO (Local Information Only).

 

What about Emby losing everything without user action? This is quite frustrating. Can't avoid it?

 

And lastly, shouldn't the backup, which I do daily, be the solution in critical moments like this?

 

Thank you

 

 

Minus having a full backup up the Emby-Server folder (Only know about Windows), primarialy the programdata folder and restoring it in most situation will get you back to the previous condition.

[Guest]

 

Even though he turned off all the trackers, it took Emby a long time, close to ten hours to reprocess the libraries, even with all the local metadata.

 

Is it possible to avoid this system failure so that the metadata does not disappear?

 

And in the case of a new failure, what is the most appropriate action we can take to recover the metadata in less time?

 

The backup, what is his function to assist in this?

 

What about manually copying any folder or file so that it is superimposed in situations like this? Is there a folder or file that we can keep in copy to replace the file in use in case this failure occurs again?

Edited June 4, 2020 by CarlosLima

[Luke 42079]

We'd have to look at the server log from the library scan to answer that. Thanks.

[Guest]

We'd have to look at the server log from the library scan to answer that. Thanks.

 

How can I send the log if it exceeds and the sending limit is only 90MB?

Thank you

 

Edited June 4, 2020 by CarlosLima

[Luke 42079]

Try zipping it up.

[Guest]

Try zipping it up.

 

Finally I managed to compress the log, which was only 5,199 KB in size, but when trying to send ...

 

 

[Guest]

It seems that the Emby system does not receive a .RAR file, so I tried the .ZIP format ...

embyserver-63726782228.zip

Edited June 4, 2020 by CarlosLima

[Happy2Play 9780]

Not that I know what the error means, but that log is full of this error. (443516 times)

2020-06-02 17: 04: 57.926 HttpServer error: ProcessAccept error
*** Error reporting ***
Version: 4.4.2.0
Command line: C: \ Users \ Administrator \ AppData \ Roaming \ Emby-Server \ system \ EmbyServer.dll
Operating system: Microsoft Windows NT 6.2.9200.0
64-bit OS: True
64-bit process: True
Interactive user: True
Runtime: file: /// C: /Users/Administrador/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
System.Environment.Version: 3.1.3
Processor count: 4
Program data path: C: \ Users \ Administrator \ AppData \ Roaming \ Emby-Server \ programdata
Application directory: C: \ Users \ Administrator \ AppData \ Roaming \ Emby-Server \ system
System.Net.Sockets.SocketException: System.Net.Sockets.SocketException (10054): An existing connection was disabled or canceled by the remote host.
in System.Net.Sockets.Socket.BeginReceive (buffer Byte [], offset Int32, size Int32, socketFlags socketFlags, AsyncCallback callback, object state)
at SocketHttpListener.SocketStream.BeginRead (Byte [] buffer, Int32 offset, Int32 count, AsyncCallback callback, object state)
in System.IO.Stream. <> c. <BeginEndReadAsync> b__48_0 (Flow stream, ReadWriteParameters args, AsyncCallback callback, object state)
in System.Threading.Tasks.TaskFactory`1.FromAsyncTrim [TInstance, TArgs] (TInstance thisRef, TArgs args, Func`5 beginMethod, Func`3 endMethod)
in System.IO.Stream.BeginEndReadAsync (byte buffer [], Int32 offset, Int32 count)
in System.IO.Stream.ReadAsync (byte buffer [], Int32 offset, Int32 count, CancellationToken cancellationToken)
in System.IO.Stream.ReadAsync (memory buffer`1, CancellationToken cancellationToken)
in System.Net.FixedSizeReader.ReadPacketAsync (stream transport, AsyncProtocolRequest request)
in System.Net.Security.SslStream.EndProcessAuthentication (result from IAsyncResult)
in System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic (result of IAsyncResult, Func`2 endFunction, action`1 endAction, task`1 promise, boolean requires Synchronization)
--- End of stack trace from the previous location where the exception was thrown ---
in SocketHttpListener.Net.HttpConnection.Init ()
in SocketHttpListener.Net.HttpEndPointListener.ProcessAccept (args SocketAsyncEventArgs)
Source: System.Net.Sockets
TargetSite: System.IAsyncResult BeginReceive (Byte [], Int32, Int32, System.Net.Sockets.SocketFlags, System.AsyncCallback, System.Object)
Send feedback
History
Saved
Community

[Luke 42079]

Are you still having an issue with this?

[Carlo 4561]

Is this a Google Drive mounted as a local hard drive with something like rclone by any chance?

In the past when I see libraries disappear like this or take a long time to reload it's almost always a non local drive that's the cause.

[Guest]

On 6/11/2020 at 3:51 PM, Luke said:

Are you still having an issue with this?

It took more than a week of hard work every day before I had metadata again. The arts are now complete, but the synopses are still many in another language and I'm still editing everything manually. The important thing is to find a way for this type of failure not to occur or that we have a less audible way to replace everything in case of another disaster, as it is quite frustrating to have to redo all the work.
Should the backup system be efficient and solve this, by the way, what is the use of emby backup for?
Thank you

[Guest]

On 6/11/2020 at 9:23 PM, cayars said:

Is this a Google Drive mounted as a local hard drive with something like rclone by any chance?

In the past when I see libraries disappear like this or take a long time to reload it's almost always a non local drive that's the cause.

Yea, in Gsuite.

[Carlo 4561]

It's not really a matter of the backup but the drives that cause this problem.   I don't mean to be a downer but I doubt this will be the last time you see something like this happen.  It's all too common that your local server can't access the drive due to Google limits and if this happens when the media is being scanned it will get dropped and have to be added back again during another scan when it finds the media again.

Then depending on your settings you may need to write data to every directory you have mounted for the NFO and graphics files so that of course takes time to do remotely and also can push the Google limits you already have issues with.

So while you can usually get a Google Drive to mostly work, it's not an ideal situation and will have issues from time to time.  It's just not going to have the reliability and constant connectivity that a local drive is going to have.

[Luke 42079]

Does this answer your question?

[Luke 42079]

@CarlosLima ?