Missing Movie Versions

[Losey 4]

I was just testing out my library and discovered some movies are missing alternate version options when viewed from a particular user.

I have three users set up on my server: a Parental Controlled user that has nothing beyond PG visible, a General Audience user with no viewing restrictions, and an Administrative user that can edit metadata. Somehow, however, the General Audience user is only displaying some versions of certain films.

For example, I have seven versions of a particular film, however only two are showing up on my General Audience user. All seven show up fine on the Parental Controlled and Administrative users.

[sa2000 674]

Could you give some actual examples, with screenshots of the tags used and what is set for the users

Also server log when the general user is not seeing all versions of a movie. Please give details, screenshot, date/time 

We have a limit of only displaying a maximum of 8 versions for a media item

There were issues before with tags and these have been addressed. I am not sure if all the fixes are in the public 4.8.10.0 release or still only in the beta (currently 4.9.31.0)

 

[Luke 42078]

@Losey?

[Losey 4]

On 11/13/2024 at 3:33 AM, sa2000 said:

Could you give some actual examples, with screenshots of the tags used and what is set for the users

Also server log when the general user is not seeing all versions of a movie. Please give details, screenshot, date/time 

We have a limit of only displaying a maximum of 8 versions for a media item

There were issues before with tags and these have been addressed. I am not sure if all the fixes are in the public 4.8.10.0 release or still only in the beta (currently 4.9.31.0)

 

12 hours ago, Luke said:

@Losey?

Apologies for the delayed response, been a crazy week with all kinds of fires to put out. Quite a lot to load up here, so I'll do my best!

Here's a screenshot of the same movie viewed from the Admin and General accounts respectively. I'll get a copy of the metadata and server log up tonight, but I have no tags on any version and all versions are the same Parental rating. These screenshots were taken just now, and I am running the latest public version of Emby.

[sa2000 674]

Thank You. So what you are saying with no tags being used, the server admin user sees 7 versions but non server admin user sees only 2. and please when providing the logs, indicate what times the screenshots are taken - so can then relate to the relevant timestamps in the logs

Please can you compare the settings for the two users - other than the server admin rights - check Access tab

Are all the versions in the same folder?

Edited November 14, 2024 by sa2000

[rbjtech 5284]

? There is no concept of 'admin rights' for the file system wrt emby - all file access will be done using whatever account loaded emby.

In other words - this is an emby database/configuration issue - it is not related to file share access if it works on one account and not from another.

[Losey 4]

8 hours ago, sa2000 said:

Thank You. So what you are saying with no tags being used, the server admin user sees 7 versions but non server admin user sees only 2. and please when providing the logs, indicate what times the screenshots are taken - so can then relate to the relevant timestamps in the logs

Please can you compare the settings for the two users - other than the server admin rights - check Access tab

Are all the versions in the same folder?

I have screenshotted each of the three users' settings and their respective parental control settings.

And yes, all the files are in the same folder, as pictured below.

Here are the server log files from immediately after checking the video through the web interface and restarting the server.

embyserver.txtembyserver-63867209436.txthardware_detection-63867209496.txt

8 hours ago, rbjtech said:

? There is no concept of 'admin rights' for the file system wrt emby - all file access will be done using whatever account loaded emby.

In other words - this is an emby database/configuration issue - it is not related to file share access if it works on one account and not from another.

Indeed, I'm not meaning to imply that. I simply have configured the "Superuser" account as being the only one allowed to edit metadata, and have it hidden.

[Losey 4]

Here's the Access page for each, and a copy of the Despecialized Edition NFO, which seems to be the only metadata file updated whenever I make an edit. The time the first screenshot I gave was taken is in the image, approximately 10:52-10:57 AM.

Star Wars- Episode Ⅳ - A New Hope (1977) - Despecialized Edition.nfo

Edited November 15, 2024 by Losey

[rbjtech 5284]

@Losey- I've raised a question with @softworkz(one of the Dev's) about Data Explorer - as currently you can't view multi-version data - and I think that is key here to understand all the metadata it has for each 'version'.   I'm thinking some versions may have different Ratings for example - they don't just inherit from the 1st version.

 

Edited November 15, 2024 by rbjtech

[sa2000 674]

5 hours ago, Losey said:

The time the first screenshot I gave was taken is in the image

I cannot see it on the 2 images you attached here

Anyway - I know now that the general user issue relates to user "Losey Home". You mention it is not restricted - but your screenshot does show ratings restriction

The log shows an error 500 for a request relating to that user account around the time of your screenshots

2024-11-14 10:52:28.546 Info Server: http/1.1 Response 500 to host1. Time: 33ms. GET http://emby_remote_ip:8096/emby/Users/cdfa1d9a82bf40b283d8dc236633cd88/Items?IncludeItemTypes=Movie&Fields=BasicSyncInfo,CanDelete,PrimaryImageAspectRatio,ProductionYear,Status,EndDate&StartIndex=0&SortBy=SortName&SortOrder=Ascending&ParentId=6&EnableImageTypes=Primary,Backdrop,Thumb&ImageTypeLimit=1&Recursive=true&Limit=0&X-Emby-Client=Emby Web&X-Emby-Device-Name=Chrome Windows&X-Emby-Device-Id=68bd5e8f-9a2b-4208-888b-1bcd0991319b&X-Emby-Client-Version=4.8.10.0&X-Emby-Token=x_secret4_x&X-Emby-Language=en-us

The log also shows that some database indexing was going on at the time 

example:

2024-11-14 10:51:53.618 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:51:53.725 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:51:56.966 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:51:57.030 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:18.320 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:18.408 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:18.522 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:22.241 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:28.515 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)

It is possible that there may be a database issue that led to the error 500 

We may get more info if you enable debug logging, do a media scan and metadata refresh and after that completes, try again and capture a fresh screenshot - noting down the time and collect the log and attach.

You could try removing the rating restrictions for that user and see if that makes a difference. 

 

 

Edited November 15, 2024 by sa2000

[ebr 16184]

5 hours ago, sa2000 said:

You mention it is not restricted - but your screenshot does show ratings restriction

Yep and I'll bet those other versions have a rating defined for them (check in metadata manager). My guess is its something like "NR" which would rate above R.

[rbjtech 5284]

19 minutes ago, ebr said:

Yep and I'll bet those other versions have a rating defined for them (check in metadata manager). My guess is its something like "NR" which would rate above R.

How ?  I mentioned this in the post above - neither metadata manager nor data Explorer list details of the multi-versions - they just list metadata for a single version.

Same as if you edit the metadata - regardless of which version you choose, it only lists the path to one ..

[Losey 4]

  

22 minutes ago, ebr said:

Yep and I'll bet those other versions have a rating defined for them (check in metadata manager). My guess is its something like "NR" which would rate above R.

Yeah, I'd have guessed that too, except all versions are showing up in the Guest user, and it's got even more strict restrictions. Something's wonky.

[Losey 4]

5 hours ago, sa2000 said:

I cannot see it on the 2 images you attached here

Anyway - I know now that the general user issue relates to user "Losey Home". You mention it is not restricted - but your screenshot does show ratings restriction

The log shows an error 500 for a request relating to that user account around the time of your screenshots

2024-11-14 10:52:28.546 Info Server: http/1.1 Response 500 to host1. Time: 33ms. GET http://emby_remote_ip:8096/emby/Users/cdfa1d9a82bf40b283d8dc236633cd88/Items?IncludeItemTypes=Movie&Fields=BasicSyncInfo,CanDelete,PrimaryImageAspectRatio,ProductionYear,Status,EndDate&StartIndex=0&SortBy=SortName&SortOrder=Ascending&ParentId=6&EnableImageTypes=Primary,Backdrop,Thumb&ImageTypeLimit=1&Recursive=true&Limit=0&X-Emby-Client=Emby Web&X-Emby-Device-Name=Chrome Windows&X-Emby-Device-Id=68bd5e8f-9a2b-4208-888b-1bcd0991319b&X-Emby-Client-Version=4.8.10.0&X-Emby-Token=x_secret4_x&X-Emby-Language=en-us

The log also shows that some database indexing was going on at the time 

example:

2024-11-14 10:51:53.618 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:51:53.725 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:51:56.966 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:51:57.030 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:18.320 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:18.408 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:18.522 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:22.241 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)
2024-11-14 10:52:28.515 Info App: Sqlite: 284 - automatic index on itemlinksexcludeinheritedtagids(ItemId)

It is possible that there may be a database issue that led to the error 500 

We may get more info if you enable debug logging, do a media scan and metadata refresh and after that completes, try again and capture a fresh screenshot - noting down the time and collect the log and attach.

You could try removing the rating restrictions for that user and see if that makes a difference. 

 

 

Changing the restrictions did not impact the visibility of the versions, however I did notice that I actually do have one item under Restrict Items With Tags that I missed when pasting together my screenshots (a stupid mistake of mine resulting from me just pasting the dropdown part over the Guest user's screenshot so it would look uniform, since I didn't notice the tag restriction).

If I remove this, the versions appear. However, I did a little testing and discovered the versions disappear no matter what tags are restricted. I'm unable to access the server to dig deeper until the evenings, but I've enabled debug logging in the meantime.

[sa2000 674]

10 minutes ago, Losey said:

If I remove this, the versions appear.

That is good progress

11 minutes ago, Losey said:

However, I did a little testing and discovered the versions disappear no matter what tags are restricted.

That needs digging into. We still have the unexplained error 500 in the log that I highlighted. So may be several issues. One being the tags.

[ebr 16184]

2 hours ago, sa2000 said:

That needs digging into. We still have the unexplained error 500 in the log that I highlighted. So may be several issues. One being the tags.

We've seen this before and I believe it came down to database corruption.

2 hours ago, rbjtech said:

How ? 

Yeah, sorry, you're right.  I was thinking that was exposed in metadata manager.  I think it may be possible for them to exist in an NFO and you can't see them but this issue is starting to look like something other than I posited.

[Losey 4]

I wouldn't be at all surprised at having some database corruption. I've only had to rebuild from scratch a couple of times early on, and more than once I've had to attempt a rescue reinstallation without deleting my old databases. I dread the inevitable future when I have to rebuild my book and game metadata since none of that is backed up in NFO files.

My library is currently running a scan, but once that finishes I'll upload the latest log files. I've had this Cache file cleanup Failed error since forever, so that's probably a contributing issue.

[Losey 4]

Here are the logs!

embyserver.txt hardware_detection-63867293355.txt

[sa2000 674]

On 16/11/2024 at 02:51, Losey said:

Here are the logs!

Thanks. What are the logs supposed to cover ? what issue? It is only for 2 minutes : 15th November from 6:47 pm to 6:49 pm

With the regards to the cache cleanup errors, they were 12 hours earlier. Suggest getting all the embyserver-xxxxxxxxxxx.txt log files as well

By default Emby Server keeps 3 days log files

 

[Luke 42078]

@Losey?

[Losey 4]

Sorry for the delay, was out of town again, but now I'm back in action. I'm not sure what I'm supposed to be looking for or what I need to do, I assume the database corruption is one of the many things I need to resolve. I've uploaded another log file "embyserver-63867998542.txt", this one has an error at: 2024-11-23 22:49:19.022 and 2024-11-22 08-05-51.116 that I was able to locate, and they both seem to be the same kind of error.

I restarted the computer and then I attempted to view and load in the movie again tonight (2024-11-24 around 7:50-7:52pm) with the "restricted" tag blocked, and allowed, and blocked again. This should be in "embyserver.txt". I have also uploaded the latest hardware_detection log I downloaded after doing this. Any suggestions on things I should try or test further?

embyserver-63867998542.txt embyserver.txt hardware_detection-63868073022.txt

[sa2000 674]

10 hours ago, Losey said:

one has an error at: 2024-11-23 22:49:19.022

The log files you attached do not cover 23rd November

10 hours ago, Losey said:

and 2024-11-22 08-05-51.116

This was the failure of the cache cleanup daily scheduled task

It failed after running for just over 6 minutes with windows indicating the system has run out of resources and cannot handle the specific action at that moment

This was the error after 6 minutes and 22 seconds

    System.IO.IOException: System.IO.IOException: Insufficient system resources exist to complete the requested service. : '\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata\cache\fanart-music\c9605929-6b48-4991-bb96-0bf460df8ee1'

Maybe your cache directory has too many items to be handled 

How much RAM on the PC?

How much free memory is there ?

Look into how many files\folders you have in the folder structure below 

\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata\cache\

You can use tools to help with that - not sure if the free Treesize Free would be ok to give a view of folders and files - download here Treesize Free

This was the error and sequence

2024-11-22 07:59:28.327 Debug TaskManager: IntervalTrigger fired for task: Cache file cleanup
2024-11-22 07:59:28.327 Debug TaskManager: Queueing task DeleteCacheFileTask
2024-11-22 07:59:28.327 Debug TaskManager: Executing Cache file cleanup

2024-11-22 08:05:51.116 Error TaskManager: Error
	*** Error Report ***
	Version: 4.8.10.0
	Command line: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system\EmbyServer.dll
	Operating system: Microsoft Windows 10.0.22631
	Framework: .NET 6.0.33
	OS/Process: x64/x64
	Runtime: /HEARTHSTONE/Videos/_backup/embyserver-win-x64/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata
	Application path: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system
	System.IO.IOException: System.IO.IOException: Insufficient system resources exist to complete the requested service. : '\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata\cache\fanart-music\c9605929-6b48-4991-bb96-0bf460df8ee1'
	   at System.IO.Enumeration.FileSystemEnumerator`1.CreateRelativeDirectoryHandle(ReadOnlySpan`1 relativePath, String fullPath)
	   at System.IO.Enumeration.FileSystemEnumerator`1.MoveNext()
	   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
	   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
	   at Emby.Server.Implementations.ScheduledTasks.Tasks.DeleteCacheFileTask.DeleteCacheFilesFromDirectory(String directory, DateTimeOffset minDateModified, IProgress`1 progress, CancellationToken cancellationToken)
	   at Emby.Server.Implementations.ScheduledTasks.Tasks.DeleteCacheFileTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
	   at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)
	Source: System.Private.CoreLib
	TargetSite: IntPtr CreateRelativeDirectoryHandle(System.ReadOnlySpan`1[System.Char], System.String)
	
2024-11-22 08:05:51.116 Debug TaskManager: Cache file cleanup Failed after 6 minute(s) and 22 seconds

Do you know when this issue started ?

The cache cleanup task is run every 24 hours

You could also look at the Windows System and Application event logs for around this time (using event viewer) to see what windows logged with regards to system resources

Lets go back to your versions issues which this topic was raised for

Do you still have an issue where versions do not show up?

Is the problem only when you use labels ? 

10 hours ago, Losey said:

I restarted the computer and then I attempted to view and load in the movie again tonight (2024-11-24 around 7:50-7:52pm) with the "restricted" tag blocked, and allowed, and blocked again. This should be in "embyserver.txt"

and ? You attempted to view and load the movie

- with the restricted tag blocked

- with the restricted tag allowed

For each case, i need the detail again and not just the log

So for each test, screenshots to show what labels are being allowed and which labels are being disallowed

The labels attached to all the versions at this moment

Sufficient to just do one test

What I am not sure about is if all the corrections for labels/versions that went into the beta got brought into the public release. Are you in a position to switch to the beta ? current is 4.9.0.32. If you do switch then you would need to stick with it until that beta release becomes public 

Edited November 25, 2024 by sa2000

[Neminem 1519]

Just out of curiosity

Is you Emby server installed on a network drive ?

Because that could cause I/O problemmes. 

Or is this your backup task, copying files to network location ?

 

2 hours ago, sa2000 said:

\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system\EmbyServer.dll

 

2 hours ago, sa2000 said:

\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata

 

[sa2000 674]

Appears to be running on Windows but installed on a network drive accessible to Windows

	Command line: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.22631

and it is curious - on a "_backup` directory

 

Edited November 25, 2024 by sa2000

[Losey 4]

On 11/25/2024 at 5:58 AM, sa2000 said:

The log files you attached do not cover 23rd November

This was the failure of the cache cleanup daily scheduled task

It failed after running for just over 6 minutes with windows indicating the system has run out of resources and cannot handle the specific action at that moment

This was the error after 6 minutes and 22 seconds

    System.IO.IOException: System.IO.IOException: Insufficient system resources exist to complete the requested service. : '\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata\cache\fanart-music\c9605929-6b48-4991-bb96-0bf460df8ee1'

Maybe your cache directory has too many items to be handled 

How much RAM on the PC?

How much free memory is there ?

Look into how many files\folders you have in the folder structure below 

\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata\cache\

You can use tools to help with that - not sure if the free Treesize Free would be ok to give a view of folders and files - download here Treesize Free

This was the error and sequence

2024-11-22 07:59:28.327 Debug TaskManager: IntervalTrigger fired for task: Cache file cleanup
2024-11-22 07:59:28.327 Debug TaskManager: Queueing task DeleteCacheFileTask
2024-11-22 07:59:28.327 Debug TaskManager: Executing Cache file cleanup

2024-11-22 08:05:51.116 Error TaskManager: Error
	*** Error Report ***
	Version: 4.8.10.0
	Command line: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system\EmbyServer.dll
	Operating system: Microsoft Windows 10.0.22631
	Framework: .NET 6.0.33
	OS/Process: x64/x64
	Runtime: /HEARTHSTONE/Videos/_backup/embyserver-win-x64/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata
	Application path: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system
	System.IO.IOException: System.IO.IOException: Insufficient system resources exist to complete the requested service. : '\\HEARTHSTONE\Videos\_backup\embyserver-win-x64\programdata\cache\fanart-music\c9605929-6b48-4991-bb96-0bf460df8ee1'
	   at System.IO.Enumeration.FileSystemEnumerator`1.CreateRelativeDirectoryHandle(ReadOnlySpan`1 relativePath, String fullPath)
	   at System.IO.Enumeration.FileSystemEnumerator`1.MoveNext()
	   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
	   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
	   at Emby.Server.Implementations.ScheduledTasks.Tasks.DeleteCacheFileTask.DeleteCacheFilesFromDirectory(String directory, DateTimeOffset minDateModified, IProgress`1 progress, CancellationToken cancellationToken)
	   at Emby.Server.Implementations.ScheduledTasks.Tasks.DeleteCacheFileTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
	   at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)
	Source: System.Private.CoreLib
	TargetSite: IntPtr CreateRelativeDirectoryHandle(System.ReadOnlySpan`1[System.Char], System.String)
	
2024-11-22 08:05:51.116 Debug TaskManager: Cache file cleanup Failed after 6 minute(s) and 22 seconds

Do you know when this issue started ?

The cache cleanup task is run every 24 hours

You could also look at the Windows System and Application event logs for around this time (using event viewer) to see what windows logged with regards to system resources

Lets go back to your versions issues which this topic was raised for

Do you still have an issue where versions do not show up?

Is the problem only when you use labels ? 

and ? You attempted to view and load the movie

- with the restricted tag blocked

- with the restricted tag allowed

For each case, i need the detail again and not just the log

So for each test, screenshots to show what labels are being allowed and which labels are being disallowed

The labels attached to all the versions at this moment

Sufficient to just do one test

What I am not sure about is if all the corrections for labels/versions that went into the beta got brought into the public release. Are you in a position to switch to the beta ? current is 4.9.0.32. If you do switch then you would need to stick with it until that beta release becomes public 

Gah, I even checked the log files before downloading them. I'll give it another try tomorrow. This issue has been ongoing since I first installed Emby. The version issue still occurs so long as I have am using the restricted tags feature. The movies work fine otherwise, and I'll get a capture of that with tomorrow's upload. I can give a try to the beta after.

On 11/25/2024 at 8:25 AM, Neminem said:

Just out of curiosity

Is you Emby server installed on a network drive ?

Because that could cause I/O problemmes. 

Or is this your backup task, copying files to network location ?

 

 

 

Yes, I have Emby running from an RN214 using an Intel NUC NUC8i3BEH Mini PC/HTPC, Intel Dual-Core i3-8109U Upto 3.6GHz, 32GB DDR4, 512GB SSD, WiFi, Bluetooth, Thunderbolt 3, 4k Support, Dual Monitor, connected through ethernet with a Netgear 8-port Router. The NAS is configured in Raid-10 with four 10 TB enterprise hard disk drives, though I plan to upgrade to 20 TB enterprise hard disk drives when I have the finances. I could probably run Emby from the NUC's storage, and run backups onto the NAS, but I couldn't be bothered to figure that out when I first set it up.

On 11/25/2024 at 8:46 AM, sa2000 said:

Appears to be running on Windows but installed on a network drive accessible to Windows

	Command line: \\HEARTHSTONE\Videos\_backup\embyserver-win-x64\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.22631

and it is curious - on a "_backup` directory

 

Odd, my NUC is running Windows 11 Pro, not Windows 10. However, that is correct, the NAS is where I have Emby installed. I put the installation in the _backup directory because I was a complete moron when I first installed it.