Jump to content

Large Library causing issues?


pir8radio
Go to solution Solved by pir8radio,

Recommended Posts

pir8radio

so I was having issues with my (what I would like to call large) library taking a long time to load, once loaded I browsed through MB3 and all looked good, I rebooted my server and now all my MB3 dashboard shows is the Trailers...     its like my library has been wiped out....   :-/     any ideas on this one?   lol,  I laugh because I have been bugging you guys so much, I promise ill stop once I get it running stable..  

I'm going to manually scan the library and hope it comes back quickly, but I'm still a bit stumped as to why it cleared out...

 

Thanks,

Link to comment
Share on other sites

Whe you reboot your Server it scans your media libaries again for updated content.

I think that's a intended behaviour.

 

When you have a great libary (like mine with over 30.000 Songs) it can take a few minutes till the scan is completeted.

As long as the scan goes on you don't have access to your libary, but it's not gone!

After the scan is completeted it's available again.

 

Maybe this is a bit anoying, when you reboot your server often.

But I never had a problem with it, because my server only hibernates.

Edited by CM000n
Link to comment
Share on other sites

pir8radio

No my media server is not scanning under scheduled or active tasks, its just plain empty.

 

I don't know if it is a "large" library but:

 

MP3's:  142,521

Music Videos: 12,528

Movies (only): 318

 

it did it again today, the server did NOT reboot this time, I woke up this morning and my library is empty again...   a scan seems to fix it, but if the MB3 server app restarts, it wipes it out again.

Link to comment
Share on other sites

aspdend

Nope it's not a large library - I would say that it is not the size that is causing issues.

 

As for the problem - I'm sorry - not come across that one before, don't remember seeing anyone raise it either - but then again, I'm no expert.

 

The first thing you need to do is post the logs

 

http://mediabrowser.tv/community/

 

Look on the right hand side of the front page - 'Having a Problem'

  • Like 2
Link to comment
Share on other sites

pir8radio

My logs vary in size up to 229 MB   I know the server kicked itself this morning...   and this was in the logs:    I have seen the "OutOfMemoryException" (unhandled logs) quite a bit, come to think about it, about the same times my library has cleared out...    i have 48 gigs of ram...   so its not a hardware issue, the machine never reports that usage is coming near that.    I'll post the logs tonight.

Exception of type 'System.OutOfMemoryException' was thrown.System.OutOfMemoryException   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.GetCompletionAction[TMethodBuilder,TStateMachine](TMethodBuilder& builder, TStateMachine& stateMachine)   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AwaitUnsafeOnCompleted[TAwaiter,TStateMachine](TAwaiter& awaiter, TStateMachine& stateMachine)--- End of stack trace from previous location where exception was thrown ---   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<ThrowAsync>b__1(Object state)   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()   at System.Threading.ThreadPoolWorkQueue.Dispatch()   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
Link to comment
Share on other sites

We are a 32bit app so we can't address anything over 2GB anyway.

 

It does sound like this is the issue and we need to see if we can figure out why it is occurring. Probably is related to the size of your music library as that is massive (and I now understand your moniker ;)).

 

If you can post the entire log, or at least the part of the log leading up to the very first occurrence of the OOM error, maybe we can find something.

Link to comment
Share on other sites

pir8radio

We are a 32bit app so we can't address anything over 2GB anyway.

 

It does sound like this is the issue and we need to see if we can figure out why it is occurring. Probably is related to the size of your music library as that is massive (and I now understand your moniker ;)).

 

If you can post the entire log, or at least the part of the log leading up to the very first occurrence of the OOM error, maybe we can find something.

 

 

I think this is what you are looking for?   Prior to this are a few DLNA errors that i normally see in my logs.


2014-10-28 01:01:58.6381 Error - Dlna: Error creating PlayTo device.
	Unable to connect to the remote server
	MediaBrowser.Model.Net.HttpException
	   at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager.<SendAsyncInternal>d__1e.MoveNext()
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
	   at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager.<SendAsync>d__7.MoveNext()
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
	   at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager.<Get>d__2.MoveNext()
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
	   at MediaBrowser.Dlna.PlayTo.SsdpHttpClient.<GetDataAsync>d__b.MoveNext()
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
	   at MediaBrowser.Dlna.PlayTo.Device.<CreateuPnpDeviceAsync>d__cc.MoveNext()
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
	   at MediaBrowser.Dlna.PlayTo.PlayToManager.<_deviceDiscovery_DeviceDiscovered>d__5.MoveNext()
	InnerException: System.Net.WebException
	Unable to connect to the remote server
	   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
	   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
	   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
	   at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager.<SendAsyncInternal>d__1e.MoveNext()
	InnerException: System.Net.Sockets.SocketException
	A socket operation was attempted to an unreachable network 169.254.5.70:49152
	   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
	   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
	
2014-10-28 01:02:51.9278 Error - Dlna: Error parsing profile xml: C:\Users\DL380e-User\AppData\Roaming\MediaBrowser-Server\config\dlna\system\Windows 8 RT.xml
	There is an error in XML document (28, 17).
	System.InvalidOperationException
	   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
	   at MediaBrowser.Common.Implementations.Serialization.XmlSerializer.DeserializeFromStream(Type type, Stream stream)
	   at MediaBrowser.Common.Implementations.Serialization.XmlSerializer.DeserializeFromFile(Type type, String file)
	   at MediaBrowser.Dlna.DlnaManager.ParseProfileXmlFile(String path, DeviceProfileType type)
	InnerException: System.OutOfMemoryException
	Exception of type 'System.OutOfMemoryException' was thrown.
	   at System.Text.StringBuilder.ToString()
	   at System.Xml.XmlTextReaderImpl.ParseText()
	   at System.Xml.XmlTextReaderImpl.ParseElementContent()
	   at System.Xml.XmlTextReaderImpl.Read()
	   at System.Xml.XmlTextReader.Read()
	   at System.Xml.XmlReader.ReadElementString()
	   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderDeviceProfile.Read21_DeviceProfile(Boolean isNullable, Boolean checkType)
	   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderDeviceProfile.Read22_Profile()
	
2014-10-28 01:06:15.0713 Error - Main: UnhandledException
	Exception of type 'System.OutOfMemoryException' was thrown.
	System.OutOfMemoryException
	   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.GetCompletionAction[TMethodBuilder,TStateMachine](TMethodBuilder& builder, TStateMachine& stateMachine)
	   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AwaitUnsafeOnCompleted[TAwaiter,TStateMachine](TAwaiter& awaiter, TStateMachine& stateMachine)
	--- End of stack trace from previous location where exception was thrown ---
	   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<ThrowAsync>b__1(Object state)
	   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
	   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
	   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
	   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
	   at System.Threading.ThreadPoolWorkQueue.Dispatch()
	   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(
Edited by Abobader
Just use code tag :)
Link to comment
Share on other sites

pir8radio

Oh I didn't know i could just post the MB3 log file directly... Thought it would require a login to view it...   Just tested looks like you can access the logs without a login...   starting to get scared about security..   :ph34r:   

 

EDIT: Removed old link

Edited by pir8radio
Link to comment
Share on other sites

Happy2Play

Oh I didn't know i could just post the MB3 log file directly... Thought it would require a login to view it...   Just tested looks like you can access the logs without a login...   starting to get scared about security..   :ph34r:   

That is a little disturbing.

Link to comment
Share on other sites

The ability to access files directly from your server is a function of your site configuration and really shouldn't have anything to do with MB.

 

Unfortunately, that log doesn't offer any clues.  The OOM is thrown out of the blue it seems.

Link to comment
Share on other sites

pir8radio

Sooooooooo, whats next?    The only thing i can see in that log (and i know nothing about MB3 lol)  is that a media scan was triggered, and i see a few errors as it hits some of my songs with really long file names, you will notice it goes in alphabetical order, the last error is in the "Z's"   it seem that right when MB3 is supposed to be ending the scan and doing whatever it does at the end of the scan is when i get the out of memory error...     or am i wearing a tin foil hat?

 

Some other info, i probably dont doubt it if i have 1 or 2 corrupt mp3's or music videos,   does MB3 handle them well?

Edited by pir8radio
Link to comment
Share on other sites

TolkienBard

Although not exactly the same, this looks very similar to the OOM error problem I have been having for a bit now. I wound up turning off all of my scheduled plug-in updates and that slowed the issue down, but has not eliminated it. When I get the chance this afternoon, I will dig out the logs and post them to my thread. Hopefully we can pin down what is causing the issue. 

Link to comment
Share on other sites

pir8radio

So i just restarted the MB3 service again today, trying to fix a search issue, and wiped my library out again...  :-(     Is there anything that i can do to help the Dev's with medium/large library issues?    Seems like my search is broken now that i added all of my media too..  worked before i added all of my library...   Willing to let dev's into my server for testing..    I don't want to have to go back to subsonic, i really like the MB project....   Ill give you more money to fix me.. lol....  :D

Link to comment
Share on other sites

I'm really not sure but with almost 150k music files I suppose even a small memory leak or inefficiency could impact you when it wouldn't others.

 

Can you bring up the task manager and watch the server process?  Watch it when it first comes up and does its scan.  Watch the memory consumption.  Where does it start?  Where does it end up after the start-up scan?  If you run another library scan can you make the memory grow without dropping back down?

Link to comment
Share on other sites

pir8radio

Ill do some trouble shooting hopefully i can find a pattern...         But this last time was really weird, i restarted the MB service,  went into MB3 and GONE no library, looked at running scheduled tasks, none running (i dont trigger the scan after a restart) MB3 is chugging along though when i go to performance monitor and look at the drive usage, its slowly but constantly reading files... in the user app folder...   but no matter how long it sits the media doesnt show int he browser view...   I load up the phone app, and it DOES show there!    I hit rescan under scheduled tasks and the scan only gets to about .5% and then everything shows up in the web interface...     i let it continue to scan and it finishes pretty quickly (1 hour or so) so i think everything is still stored in the database, it just seems when i restart the app the web interface is blank until i start a rescan..   hopefully that helps until i can do some more digging.

Edited by pir8radio
Link to comment
Share on other sites

You said "service".  Are you actually running as a service?  If so, are you sure 1) you configured the user permissions properly and 2) that all the resources are available when the service starts?

Link to comment
Share on other sites

pir8radio

I am running as a services, yep permissions set as an admin account rather than system account on the service, and 2, i assume so, but when i restarted the service, it was already running previously i just restarted that single service..  Ill switch it back to the app starting itself just so there are no weird things with the service going on..

Link to comment
Share on other sites

With the small section of the log posted above there is an ip address of 169.254.5.70:49152 that is not a valid IP, not sure what device that IP is for but that is an IP in limbo that the MBS is seeing.

You need to look into that. A reboot of all your network might be in order, Modem, router, PC's, devices.

 

Also with that many Music files you can easily have non US characters or other special characters in one or more files and that could be causing a problem. Check your logs for errors.

I have had files from the internet that the the TAGs were so screwed up that even removing then and repalce them would not fix it, the files had to be reencode to fix them so the tags would be normal.

Link to comment
Share on other sites

pir8radio

Yea I don't know where Media browser is getting that IP, its for a DLNA device that I use, and is pingable, and playable at a legit 10.0.X.X address...

 

EBR here is my memory usage after a reboot...

 

5453152ec4eda_Capture.png

Edited by pir8radio
Link to comment
Share on other sites

pir8radio

I may be legitimately using that much memory depending on how media browser uses the cache stuff in the app data folder...  For example, here are the properties for my APPDATA\roaming\mediabrowser-server folder:   (how bout making me a 64 bit version  :mellow:)

 

54531c5d4e168_folderinfo.png

Edited by pir8radio
Link to comment
Share on other sites

With about 2,500 movies, another 275 TV shows, and about 20,000 songs on a quad core Xeon with 8gb, it takes roughly 30 minutes after a restart of the MediaBrowser server to settle down and become fully responsive. During that time, it doesn't always show all of the media, but it does come back once the process is complete.

Link to comment
Share on other sites

pir8radio

I think i would love if my scan finished in 30 mins to an hour.. lol..   I think yours sounds about correct though.  My problem is the scan usually doesnt complete,  it loads and loads the info from the app data folder then uses all of the memory.. and craps out...  however if i make sure no one logs onto the MB server while the scan is happening....  it consumes just under 1.4gigs of ram and maintains that....  once the scan is done, the memory usage for MB goes down to about 600 meg and stays there...   so far my fix is to not let the app do "other things" while scanning...   but my library continues to grow every day...  by a hundred or so songs.. so, im a bit worried how far i can go?

Link to comment
Share on other sites

Holy crap 1.3 gig on start-up?

 

Well, then yeah I guess it probably wouldn't take too much to push you to where we can run out of memory.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...