Jump to content

Bad music library performance when MusicBrainz is enabled

fc7

By fc7
in Linux

 Share
Go to solution Solved by Luke,

Recommended Posts

fc7

fc7 123

Posted
Posted (edited)

I detected this several weeks ago but I only had time to report it now. I'm running Emby 5930 (latest stable).

 

Can't remember the exact Emby version when the problem started but since some time now (maybe two months) my library scans went from around 2:30mins to around 12-14 mins.

 

Today I was checking the logs and I saw tons of exceptions while contacting musicbrainz.org since the site is returning a 503 error for many requests from Emby:

2016-04-19 21:32:01.4539 Info HttpClient: HttpClientManager GET: http://www.musicbrainz.org/ws/2/release/?query="(2001)+Just+Push+Play" AND arid:3d2b98e5-556f-4451-a3ff-c50ea18d57cb
2016-04-19 21:32:01.6613 Error HttpClient: Error getting response from http://www.musicbrainz.org/ws/2/release/?query="(2001)+Just+Push+Play" AND arid:3d2b98e5-556f-4451-a3ff-c50ea18d57cb
	*** Error Report ***
	Version: 3.0.5930.0
	Command line: /usr/lib/emby-server/bin/MediaBrowser.Server.Mono.exe -programdata /var/lib/emby-server -restartpath /usr/lib/emby-server/restart.sh
	Operating system: Unix 3.10.0.327
	Processor count: 4
	64-Bit OS: True
	64-Bit Process: True
	Program data path: /var/lib/emby-server
	Mono: 4.2.1 (Stable 4.2.1.102/6dd2d0d Wed Jan 13 16:44:34 UTC 2016)
	Application Path: /usr/lib/emby-server/bin/MediaBrowser.Server.Mono.exe
	One or more errors occurred.
	System.AggregateException
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager+<SendAsyncInternal>c__async4.MoveNext () <0x4154ae50 + 0x01055> in <filename unknown>:0 
	InnerException: System.Net.WebException
	The remote server returned an error: (503) Service Unavailable.
	  at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) <0x415e0320 + 0x001a3> in <filename unknown>:0 
	  at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (IAsyncResult iar, System.Func`2 endFunction, System.Action`1 endAction, System.Threading.Tasks.Task`1 promise, Boolean requiresSynchronization) <0x7f9d6ded3dd0 + 0x00088> in <filename unknown>:0
2016-04-19 21:32:01.6626 Error App: Error in MusicBrainz
	*** Error Report ***
	Version: 3.0.5930.0
	Command line: /usr/lib/emby-server/bin/MediaBrowser.Server.Mono.exe -programdata /var/lib/emby-server -restartpath /usr/lib/emby-server/restart.sh
	Operating system: Unix 3.10.0.327
	Processor count: 4
	64-Bit OS: True
	64-Bit Process: True
	Program data path: /var/lib/emby-server
	Mono: 4.2.1 (Stable 4.2.1.102/6dd2d0d Wed Jan 13 16:44:34 UTC 2016)
	Application Path: /usr/lib/emby-server/bin/MediaBrowser.Server.Mono.exe
	One or more errors occurred.
	MediaBrowser.Model.Net.HttpException
	  at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager+<SendAsyncInternal>c__async4.MoveNext () <0x4154ae50 + 0x01833> in <filename unknown>:0 
	--- End of stack trace from previous location where exception was thrown ---
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager+<SendAsync>c__async1.MoveNext () <0x415453f0 + 0x00215> in <filename unknown>:0 
	--- End of stack trace from previous location where exception was thrown ---
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager+<Get>c__async0.MoveNext () <0x415ca1c0 + 0x00174> in <filename unknown>:0 
	--- End of stack trace from previous location where exception was thrown ---
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Providers.Music.MusicBrainzAlbumProvider+<GetMusicBrainzResponse>c__async5.MoveNext () <0x41e95c00 + 0x0068e> in <filename unknown>:0 
	--- End of stack trace from previous location where exception was thrown ---
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Providers.Music.MusicBrainzAlbumProvider+<GetReleaseResult>c__async2.MoveNext () <0x41eedab0 + 0x0020f> in <filename unknown>:0 
	--- End of stack trace from previous location where exception was thrown ---
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Providers.Music.MusicBrainzAlbumProvider+<GetMetadata>c__async1.MoveNext () <0x41ee37b0 + 0x00533> in <filename unknown>:0 
	--- End of stack trace from previous location where exception was thrown ---
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Providers.Manager.MetadataService`2+<ExecuteRemoteProviders>c__async8[TItemType,TIdType].MoveNext () <0x41e17950 + 0x00686> in <filename unknown>:0 
	InnerException: System.AggregateException
	One or more errors occurred.
	  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x7f9d6de48790 + 0x00029> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0x7f9d6de46760 + 0x000b3> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0x7f9d6de466c0 + 0x00093> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0x7f9d6de46670 + 0x0003a> in <filename unknown>:0 
	  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () <0x7f9d6de46dd0 + 0x00017> in <filename unknown>:0 
	  at MediaBrowser.Common.Implementations.HttpClientManager.HttpClientManager+<SendAsyncInternal>c__async4.MoveNext () <0x4154ae50 + 0x01055> in <filename unknown>:0 
	InnerException: System.Net.WebException
	The remote server returned an error: (503) Service Unavailable.
	  at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) <0x415e0320 + 0x001a3> in <filename unknown>:0 
	  at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (IAsyncResult iar, System.Func`2 endFunction, System.Action`1 endAction, System.Threading.Tasks.Task`1 promise, Boolean requiresSynchronization) <0x7f9d6ded3dd0 + 0x00088> in <filename unknown>:0

This is just an example for one request but my music library is around 20K songs so there are many more in the logs.

 

I try to disable MusicBrainz as a metadata source and the library scans are back to "normal", taking just 2:30mins.

 

Is anyone else also seeing this behavior or is just me?

I'm not familiar with MusicBrainz, is it like opensubtitles that requires to have a user account to query the site?

 

Cheers.

Edited by fc7
  • Solution
Luke Emby Team

Luke 40072

Posted
  • Solution
Posted

we have to throttle music brainz pretty severely in order to stay under their request limit. there's not much we can do about that, so it ends up creating a small pause between requests.

  • Like 1
fc7

fc7 123

Posted
  • Author
Posted

Thanks for the reply Luke.

 

It's crystal clear now.

 

 

Sent from my iPhone using Tapatalk

  • 2 weeks later...
tnicks

tnicks 2

Posted
Posted

we have to throttle music brainz pretty severely in order to stay under their request limit. there's not much we can do about that, so it ends up creating a small pause between requests.

 

I have a local MusicBrainz server. Is there a way to point Emby to use that, rather than the public instance that is throttled? Being able to use a custom server that isn't throttled would be great for those of us with large collections.

  • Like 1
Luke Emby Team

Luke 40072

Posted
Posted

At the moment there is no configuration for that.

  • 1 month later...
Habitual_Sinner

Habitual_Sinner 5

Posted
Posted

I also have a local MusicBrainz server and quite a large music collection to go with it (around 12000 albums)

Are there any plans to implement this feature in the future? Is there a config file in Emby that would allow us to substitute the MusicBrainz public URL with a local IP?

fc7

fc7 123

Posted
  • Author
Posted

I also have a local MusicBrainz server and quite a large music collection to go with it (around 12000 albums)

Are there any plans to implement this feature in the future? Is there a config file in Emby that would allow us to substitute the MusicBrainz public URL with a local IP?

Are you running both the slave DB and also the search-server on your local musicbrainz server?

 

I'm asking because as far as I know unless you are running the search-server too all the search requests will end up hitting the musicbrainz public search-server which will result in throttling too.

 

 

Sent from my iPad using Tapatalk

Habitual_Sinner

Habitual_Sinner 5

Posted
Posted

I'm running both, I converted a virtualbox image to run as a standalone vm on a VMware host about a year ago when I started getting serious about ripping my vinyl collection to flac

 

Sent from my LG-H815 using Tapatalk

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...