smokey7722 12 Posted October 22, 2013 Share Posted October 22, 2013 I recently moved all my systems over the MediaBrowser 3 and am very happy. Lots of little kinks here and there but I am slowly working through it all. Todays issue which I haven't found any reference for online or in the forums is while attempting to run the "Refresh people" scheduled task, I see a large amount of errors in my log. The error repeats over and over again for each person's name. I can upload a full log but as far as I could see there was no additional entries related to this error so I figured I would try this first to see if anyone had any ideas. ________________________________________________________________________________________________________________ 2013-10-22 15:23:25.6331, Error, App, Error validating IBN entry Gary Goddard An attempt was made to create more links on a file than the file system supports. System.IO.IOException at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.Directory.InternalCreateDirectory(String fullPath, String path, Object dirSecurityObj, Boolean checkHost) at System.IO.Directory.InternalCreateDirectoryHelper(String path, Boolean checkHost) at System.IO.Directory.CreateDirectory(String path) at MediaBrowser.Server.Implementations.Library.LibraryManager.CreateItemByName[T](String path, String name) at MediaBrowser.Server.Implementations.Library.LibraryManager.GetItemByName[T](String path, String name) at MediaBrowser.Server.Implementations.Library.LibraryManager.<ValidatePeople>d__3c.MoveNext()________________________________________________________________________________________________________________ My configuration is: - MB Server 3.0.5031.21343 - Samba 3.6.18 The MB Server has access to all of the file shares via SMB being served up by a FreeBSD server. All of the "media" folders, being TV Shows or Movies or Music, are set to ReadOnly access however the ImagesByName path is set to a share that has ReadWrite access on my file server and being accessed by SMB as well. The interesting thing is I had folders being created early on however it has since stopped (current qty of folders in the People folder is 32,765) so any name thats displaying an error, it will not create a folder for. Looking at the logs for Samba I see: open_directory: unable to create People/Gary Goddard. Error was NT_STATUS_TOO_MANY_LINKS So although I know this doesn't seem to be an MB issue, I was hoping someone may have attempted to put the ImagesByName folder on a samba share before in the past and maybe seen something like this? I haven't been able to find much at all online about anyone else seeing an error like this at all. Temporarily I have moved the ImagesByName folder back to the local disk of the VM I am using for the MB Server however I would really rather keep everything on the file server as I had done previously with MB2. Any thoughts would be appreciated. Link to comment Share on other sites More sharing options...
Tikuf 663 Posted October 22, 2013 Share Posted October 22, 2013 http://superuser.com/questions/66331/what-is-the-maximum-number-of-folders-allowed-in-a-folder-in-linux Please file a bug on git as this will likely come and bite us at a later date Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 22, 2013 Author Share Posted October 22, 2013 Will do, thanks for the quick response! If I recall didn't MB2's ImagesByName folder have People separated alphabetically by name? Example: <ImagesByName>/People/A/<all names starting with A> <ImagesByName>/People/B/<all names starting with B> <ImagesByName>/People/C/<all names starting with C> I could be wrong but I recall something like that in the past. And in theory that should resolve the contraint as hopefully there would not be that many names under each letter? Link to comment Share on other sites More sharing options...
Luke 37186 Posted October 23, 2013 Share Posted October 23, 2013 Wow, yea. This is a big breaking change you're suggesting, and a lot of people are going to have to be notified, like MCM and Metabrowser for example. But given our goal to support every platform around, it looks like it needs to be done. I think I'll do it in a way that will allow existing users to keep their same convention, but all new installs will use the subfolders. Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 23, 2013 Author Share Posted October 23, 2013 Wow, yea. This is a big breaking change you're suggesting, and a lot of people are going to have to be notified, like MCM and Metabrowser for example. But given our goal to support every platform around, it looks like it needs to be done. I think I'll do it in a way that will allow existing users to keep their same convention, but all new installs will use the subfolders. Makes sense. One thing though - so in my case for example I am not a new install but I would need the new functionality so there would need to be a way to force it I guess? I'd prefer to move the data back to my file server rather than keeping it on the VM. Link to comment Share on other sites More sharing options...
ebr 14949 Posted October 23, 2013 Share Posted October 23, 2013 Makes sense. One thing though - so in my case for example I am not a new install but I would need the new functionality so there would need to be a way to force it I guess? I'd prefer to move the data back to my file server rather than keeping it on the VM. In that situation, you would also need a conversion routine... Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 23, 2013 Author Share Posted October 23, 2013 In that situation, you would also need a conversion routine... Could just as easily just rerun the scheduled task and let it start over rather than converting thats there. So steps would be change the path via the web interface, then manually kick off the task - just like it is now. Since it wouldn't find anything in the destination folder (I would not move my current data) it would in theory be doing new and would just need to make sure its using the new method vs old. Link to comment Share on other sites More sharing options...
ebr 14949 Posted October 23, 2013 Share Posted October 23, 2013 Could just as easily just rerun the scheduled task and let it start over rather than converting thats there. So steps would be change the path via the web interface, then manually kick off the task - just like it is now. Since it wouldn't find anything in the destination folder (I would not move my current data) it would in theory be doing new and would just need to make sure its using the new method vs old. That would work if you pointed it to an entirely new location but it would duplicate a lot of work. If you left your people folder intact, the new method will fail because we wouldn't be able to create the new "A", "B", "C" etc. folders because you've already hit the file system max. Plus, even if you deleted 26 people to allow that, you'd end up with a mish-mash combination of the two methods. Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 23, 2013 Author Share Posted October 23, 2013 That would work if you pointed it to an entirely new location but it would duplicate a lot of work. If you left your people folder intact, the new method will fail because we wouldn't be able to create the new "A", "B", "C" etc. folders because you've already hit the file system max. Plus, even if you deleted 26 people to allow that, you'd end up with a mish-mash combination of the two methods. True. Thats why instead of a conversion I was thinking a fresh start, although its a lot of duplicate work. When I removed the smb path and used the local machines path, it ran through the script in < 5 minutes to create all the data. And this would only need to be done once to get it setup, and would guarantee its setup right whereas a conversion can have all sorts of issues. Since theres no custom data in there that can't easily be replicated, a fresh start may be the easiest to code and guarantee its setup right in the end. I'll butt out in the design though as you guys know what you are doing. I do QA and various software design for work so its just my habit of lending my opinion on ways to correct issues. Sorry Link to comment Share on other sites More sharing options...
ebr 14949 Posted October 23, 2013 Share Posted October 23, 2013 No, good feedback. A fresh start is definitely easier on everyone except the network . Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 23, 2013 Author Share Posted October 23, 2013 No, good feedback. A fresh start is definitely easier on everyone except the network . Very true and since its only a few minutes of heavy load done once, its not that bad. Not like its a full database rebuild that requires hour after hour of hitting it hard. Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 23, 2013 Author Share Posted October 23, 2013 Wow that was fast, the bug on git has already got code changes, applied and closed! Wow you guys work fast! Now the question is when and how can I pull down the changes to test it out? Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 (edited) Hey so just wanted to give you guys an update... I just upgraded to the new version today and while its scanning my library after the update I changed the path for the ImagesByName and set the flag via the xml. Right now with the network path (I am about 44% complete): From inside the people folder: ls | wc -l 31 ls -R | wc -l 122511 So definitely looking great! Thanks for the quick turnaround on this! Edited October 31, 2013 by smokey7722 Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 Hey so just wanted to give you guys an update... I just upgraded to the new version today and while its scanning my library after the update I changed the path for the ImagesByName and set the flag via the xml. Right now with the network path (I am about 44% complete): From inside the people folder: ls | wc -l 31 ls -R | wc -l 122511 So definitely looking great! Thanks for the quick turnaround on this! Looks like it finally just finished. Has something like 230,000 folders within People recursively but more importantly it looks like the largest folder, J has close to 9000 folders in its top level. Long term I am going to guess I may run into the same problem there but hopefully by that point the file system restriction is gone. Link to comment Share on other sites More sharing options...
crashkelly 62 Posted October 31, 2013 Share Posted October 31, 2013 (edited) I adopted the new people standard as well. I just deleted the contents of my existing people folder, made the change in the system.xml file, and then hit the "Refresh People" task and away it went. Took hours and hours to get them all, but I did it right before bed so no issues with user usage at the same time. Cheers set the value EnablePeoplePrefixSubFolders to true. Original Post: http://mediabrowser.tv/community/index.php?/topic/1594-process-to-move-to-new-people-folder-structure/ Edited October 31, 2013 by crashkelly Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 After upgrading to 3.0.5050.37565 last night - Looking at the ImagesByName folder now, it looks like the new prefix subfolders are created, and the persons name below that however all of my folders are empty. I've manually run the "Refresh people" task and its also now run by itself overnight. My steps after the upgrade were: - Upgrade - Edit the system.xml and set the new EnablePeoplePrefixSubFolders option to True - Via the web client, go to Default Media Library -> Settings and enable "Use custom images by name path" - Provide a read/write network path (this was a completely new path with no content in it so MB3 had to create the structure and populate all) I then saved and kicked off the scheduled task and it went through the hundreds/thousands of people creating their folders as expected. However each and every folder is empty - no images. I then looked at the local server path for MB3 and noticed the same thing. That there is a ton of folders, but no files/images saved in them. Am I missing something really obvious here? Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 your internet provider settings. I posted them up in my other thread you were helping on. Right now I have none disabled. The way I read the options for metadata was that there was no setting to enable/disable ImagesByName downloading specifically but it also looked like even if I had other metadata disabled (as I do) this portion would still operate. Copied from the other thread to save having to go back for reference: From the Metadata page: Basics: Download metadata from internet (unchecked) Save metadata within media folders (unchecked) Preferred language: English Country: US Enable automatic tmdb updates (unchecked) Enable automatic tvdb updates (unchecked) Images: Image saving convention: Standard - MB3/MB2 Max number of backdrops per item: 10 Enable additional image downloading: enabled only for Music Artists and Music Albums. Disabled for Movies, TV Series and TV Seasons Enable video image extraction (checked) Advanced: Disable internet providers for: (all unchecked) Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 (edited) This option will disable everything Download metadata from internet The advanced page will also turn off for specific types, such as Person. Ah ok, that explains that. So if I want to enable it for the People folder only then I need to enable download metadata from internet and DISABLE all internet providers but Person? Ultimately I would like anything in the ImagesByName folder to be downloaded. Right now I have folders for artists, Genre, MusicGenre, People, Studio so I would guess those need to be enabled. Since I already have the metadata downloaded for the TV/Movies already, I want to make sure those are disabled at least. Edited October 31, 2013 by smokey7722 Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 yes Ok I have it enabled with everything disabled but Person and Studio. Its scanning and downloading now. Will post back with final results. Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 Ok I have it enabled with everything disabled but Person and Studio. Its scanning and downloading now. Will post back with final results. So this is interesting... 2013-10-31 10:01:17.8159, Debug, ProviderManager, Running TmdbPersonProvider for \\randal_rw.quickstop.local\MediaBrowserBackEnd\People\V\Valerie Leon 2013-10-31 10:01:18.1138, Info, App, HttpClientManager.Get url: http://api.themoviedb.org/3/search/person?api_key=f6bd687ffa63cd282b6ff2c6877f2669&query=Valerie+Leon 2013-10-31 10:01:18.5610, Info, App, HttpClientManager.Get url: http://api.themoviedb.org/3/person/95099?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=credits,images 2013-10-31 10:01:18.6919, Debug, TmdbPersonProvider, TmdbPersonProvider downloaded and saved information for Valerie Leon 2013-10-31 10:01:19.0278, Info, App, HttpClientManager.Get url: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/original/idolS3hDtXVqAwncYLb1hynCMmV.jpg 2013-10-31 10:01:19.3227, Debug, TmdbPersonProvider, TmdbPersonProvider downloaded and saved image for Valerie Leon 2013-10-31 10:01:19.3227, Debug, ProviderManager, Running TvdbPersonImageProvider for \\randal_rw.quickstop.local\MediaBrowserBackEnd\People\V\Valerie Leon According to that log entry, it looks like it found Valerie Leon, and saved her image to \\randal_rw.quickstop.local\MediaBrowserBackEnd\People\V\Valerie Leon as it should. However the contents of that folder is just a single "person.xml" and contains: <?xml version="1.0" encoding="utf-8" standalone="yes"?> <Item> <Added>10/30/2013 5:18:46 PM</Added> <LockData>false</LockData> <Overview><![CDATA[From Wikipedia, the free encyclopedia. Valerie Leon (born 12 November 1943) is an English actress who had roles in a number of high profile British film franchises, including the Carry On series. Description above from the Wikipedia article Valerie Leon, licensed under CC-BY-SA, full list of contributors on Wikipedia.]]></Overview> <LocalTitle>Valerie Leon</LocalTitle> <BirthDate>1943-11-12</BirthDate> <TMDbId>95099</TMDbId> <PlaceOfBirth>London</PlaceOfBirth> </Item> Its still only 16% done the refresh people task so I will wait until its complete before I draw any conclusions but I figured I would post this up as it seemed odd that although it said it downloaded the image, the only thing there is the xml and no image. Link to comment Share on other sites More sharing options...
Luke 37186 Posted October 31, 2013 Share Posted October 31, 2013 the save local setting being disabled might be causing the image to go to our internal data folder. it's supposed to override for people but perhaps that's not working. Link to comment Share on other sites More sharing options...
smokey7722 12 Posted October 31, 2013 Author Share Posted October 31, 2013 the save local setting being disabled might be causing the image to go to our internal data folder. it's supposed to override for people but perhaps that's not working. My settings right now are: From the Metadata page: Basics: Download metadata from internet (checked) Save metadata within media folders (unchecked) Preferred language: English Country: US Enable automatic tmdb updates (unchecked) Enable automatic tvdb updates (unchecked) Images: Image saving convention: Standard - MB3/MB2 Max number of backdrops per item: 10 Enable additional image downloading: enabled only for Music Artists and Music Albums. Disabled for Movies, TV Series and TV Seasons Enable video image extraction (checked) Advanced: Disable internet providers for: (all checked EXCEPT Person and Studio) Link to comment Share on other sites More sharing options...
Luke 37186 Posted November 1, 2013 Share Posted November 1, 2013 After upgrading to 3.0.5050.37565 last night - Looking at the ImagesByName folder now, it looks like the new prefix subfolders are created, and the persons name below that however all of my folders are empty. I've manually run the "Refresh people" task and its also now run by itself overnight. My steps after the upgrade were: - Upgrade - Edit the system.xml and set the new EnablePeoplePrefixSubFolders option to True - Via the web client, go to Default Media Library -> Settings and enable "Use custom images by name path" - Provide a read/write network path (this was a completely new path with no content in it so MB3 had to create the structure and populate all) I then saved and kicked off the scheduled task and it went through the hundreds/thousands of people creating their folders as expected. However each and every folder is empty - no images. I then looked at the local server path for MB3 and noticed the same thing. That there is a ton of folders, but no files/images saved in them. Am I missing something really obvious here? your internet provider settings. Link to comment Share on other sites More sharing options...
smokey7722 12 Posted November 1, 2013 Author Share Posted November 1, 2013 the save local setting being disabled might be causing the image to go to our internal data folder. it's supposed to override for people but perhaps that's not working. I just took a look as well at the local directory of the MB3 server and neither the old folder path nor the new path has anything in the Valerie Leon folder from the above example. Link to comment Share on other sites More sharing options...
Luke 37186 Posted November 1, 2013 Share Posted November 1, 2013 This option will disable everything Download metadata from internet The advanced page will also turn off for specific types, such as Person. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now