NFO files

[thwaller 2]

Hello-

I am a new user to Emby, having experience with Kodi prior, so I am not really new to the media server concept. I use tMM (Tiny Media Manager for those who are not aware) for my media management.

 

After installing Emby, I had a few "disasters" by which Emby created additional NFO files, in the format of <movie_name>.nfo. This would be fine except I was using the method of movie.nfo as each movie has its own folder. I wonder here if there is a way to change that in Emby, the format it uses? Regardless, I changed my format and converted all my NFO files to match the movie name. But now I have some questions. I edit my NFO files with care in tMM. So, I ask:

 

1) Can I set Emby to read only NFO files?

2) Only write NFO files if it finds an entity, movie or TV episode, without an existing NFO?

3) If there is data that Emby would like to store, have it stored in an Emby specific NFO or other file format? So you would have the NFO core + additional data from the other file?

4. Is it possible to set Emby to read from NFO and write to XML? So it can copy the data from tMM and make any changes it wants to its own copy without change to the tMM (original) data?

 

I hope you can see and understand what I am doing here. I have some concern of poor data writes to the NFO files. I am not yet fully aware of how Emby works, but when I open tMM, it does not rescan all the NFO files for changes, which I would not expect or want it to unless asked. But this means that should Emby modify one, tMM is not aware, so if I make a change in tMM, the changes Emby made are overwritten ... well ... possibly. I am asking these questions so I can wrap my head around this and determine potential issues.

 

For those that wish, or need, to know...

I am using Version 4.1.0.3 beta on Linux. Ubuntu 18.04.1 LTS, 64-bit, kernel 4.15.0-44 with MATE 1.20.1 DE.

[wickedlemon 6]

I'd be interested to know this too. My limited experience trying to get Emby running properly shows it likes re-writing to nfo files even when there is nothing to change.

[Luke 37251]

I'd be interested to know this too. My limited experience trying to get Emby running properly shows it likes re-writing to nfo files even when there is nothing to change.

 

When records get saved into the emby database, metadata also gets written. That is why you sometimes see this. Please note that any custom nodes in the nfo file will be preserved.

[Luke 37251]

 

1) Can I set Emby to read only NFO files?

2) Only write NFO files if it finds an entity, movie or TV episode, without an existing NFO?

3) If there is data that Emby would like to store, have it stored in an Emby specific NFO or other file format? So you would have the NFO core + additional data from the other file?

4. Is it possible to set Emby to read from NFO and write to XML? So it can copy the data from tMM and make any changes it wants to its own copy without change to the tMM (original) data?

 

Yes, no, no, yes. The last one can be accomplished using the xml metadata plugin. But disclaimer, it is considered legacy at this point and is only community maintained.

[thwaller 2]

Yes, no, no, yes. The last one can be accomplished using the xml metadata plugin. But disclaimer, it is considered legacy at this point and is only community maintained.

 

Ok, so I gather from your reply that #1 appears to be the only logical yes. As I am new to Emby, it does not make much sense to start building with obsolete pieces. So, what is the procedure for #1? I have read a few posts discussing this, but they appear to be related to an earlier version of Emby or in other ways not applying. I guess it makes sense to have it all or nothing, as a compromise in between could be hard to make and manage. One alternative, given a clear procedure of turning off media management in Emby, would be to create a separate library for read only media and one for a library in which Emby can alter. The reason for such a thing is the difference between established media and new media. Once media has gone through a cleaning process in a media manager, it would then get moved to a "read only" Emby library. Does this make sense to you, and seem workable with Emby without causing issues?

[Luke 37251]

You would simply turn off nfo metadata saving. The downside of this though that if you ever use the emby metadata editor to make changes, they will not be written to the nfo file, and they could end up eventually getting lost altogether the next time the server re-imports data from the nfo. That could happen if the nfo ever changes or if you refresh metadata manually on something.

[thwaller 2]

You would simply turn off nfo metadata saving. The downside of this though that if you ever use the emby metadata editor to make changes, they will not be written to the nfo file, and they could end up eventually getting lost altogether the next time the server re-imports data from the nfo. That could happen if the nfo ever changes or if you refresh metadata manually on something.

 

By "turn off nfo metadata saving" you mean to unclick the box for the NFO plugin, yes?

 

There is danger in allowing 2 systems to manage the NFO. The only way that can work without concern is to have each system refresh all of its data on each run, and that is somewhat unrealistic on large libraries. The NFO is essentially a database, so the same concepts of database management apply here. Do you have any documentation on the interaction of Emby to the NFO? Meaning something like API documentation, where it is outlined on exactly which requests can / could be executed and what triggers them... this would give me a picture of what is happening, or could happen, and which parts are controllable via settings and which are not.

 

I have not yet looked, but I assume that all of the data contained in the NFO is stored in the Emby database, and that the NFO file is basically a copy of the data in the database? As long as the database is able to be queried in a normal way, the Emby changes are easy enough to identify and export and merge when / if needed.

[wickedlemon 6]

When records get saved into the emby database, metadata also gets written. That is why you sometimes see this. Please note that any custom nodes in the nfo file will be preserved.

 

Interesting to know. So if all your data is already stored in nfos an initial scan will involve a lot of nfo writing when virtually none is actually required.

[alexpaton 10]

The only purpose for the nfo files in TMM is to enable quicker (and better) scraping in Kodi.  TMM doesn't use them at all, only creates them.  The only danger here is that changes to the nfo FROM emby, get overwritten BY TMM, if you run TMM in the future; however, for previously scraped items in TMM, it shouldn't rewrite them, unless you specifically tell it to.  In other words, TMM only uses the nfo file when the movie or tv series is initially identified and scraped.

 

I also ended up using TMM, on my path to Emby, and found that its scraping knocks the socks off the competition.

[thwaller 2]

The only purpose for the nfo files in TMM is to enable quicker (and better) scraping in Kodi.  TMM doesn't use them at all, only creates them.  The only danger here is that changes to the nfo FROM emby, get overwritten BY TMM, if you run TMM in the future; however, for previously scraped items in TMM, it shouldn't rewrite them, unless you specifically tell it to.  In other words, TMM only uses the nfo file when the movie or tv series is initially identified and scraped.

 

I also ended up using TMM, on my path to Emby, and found that its scraping knocks the socks off the competition.

 

Thanks for that information. I understand that tMM uses its own database for its data, I believe the same is true for Emby. The NFO to be a portable meta data file, as you stated, for a preset scrapping vs doing a redundant lookup when it is already done and made correct. Where my concern is, let's say I am in tMM and find a movie is missing artwork, and I get that artwork. I would then rewrite the NFO file. Thus, anything changed by Emby is wiped, well, potentially. The NFO file is assumed to be an export of tMM's data, so if it is not and it is not imported in, it will be made to match tMM. Same the other way around, if tMM makes a change, a write from Emby would write from its database to the NFO, overwriting anything tMM may have changed.

 

I think I see enough of what is happening here to answer my questions now. I was hoping to find some controls on the handling of the NFO reading and writing process. I have had too many bad experiences blindly trusting code to do writes and syncs, and have the bug reports to prove that. I have come to distrust behind the scenes data manipulations in media centers and players when external services and/or softwares are involved. For me here, this is tMM + Emby + Trakt ... I need to see what is happening or at least be able to audit the data.

[Luke 37251]

By "turn off nfo metadata saving" you mean to unclick the box for the NFO plugin, yes?

 

There is danger in allowing 2 systems to manage the NFO. The only way that can work without concern is to have each system refresh all of its data on each run, and that is somewhat unrealistic on large libraries. The NFO is essentially a database, so the same concepts of database management apply here. Do you have any documentation on the interaction of Emby to the NFO? Meaning something like API documentation, where it is outlined on exactly which requests can / could be executed and what triggers them... this would give me a picture of what is happening, or could happen, and which parts are controllable via settings and which are not.

 

I have not yet looked, but I assume that all of the data contained in the NFO is stored in the Emby database, and that the NFO file is basically a copy of the data in the database? As long as the database is able to be queried in a normal way, the Emby changes are easy enough to identify and export and merge when / if needed.

 

Well yes, both api and db access are possible.

[alexpaton 10]

Thanks for that information. I understand that tMM uses its own database for its data, I believe the same is true for Emby. The NFO to be a portable meta data file, as you stated, for a preset scrapping vs doing a redundant lookup when it is already done and made correct. Where my concern is, let's say I am in tMM and find a movie is missing artwork, and I get that artwork. I would then rewrite the NFO file. Thus, anything changed by Emby is wiped, well, potentially. The NFO file is assumed to be an export of tMM's data, so if it is not and it is not imported in, it will be made to match tMM. Same the other way around, if tMM makes a change, a write from Emby would write from its database to the NFO, overwriting anything tMM may have changed.

 

I think I see enough of what is happening here to answer my questions now. I was hoping to find some controls on the handling of the NFO reading and writing process. I have had too many bad experiences blindly trusting code to do writes and syncs, and have the bug reports to prove that. I have come to distrust behind the scenes data manipulations in media centers and players when external services and/or softwares are involved. For me here, this is tMM + Emby + Trakt ... I need to see what is happening or at least be able to audit the data.

Yeah, it's a potential minefield.  I believe that I have proved to myself how much influence each has over the content of the nfo files, though.  I am not positive that I can explain it, but will try.  

 

I found a tv series which was mis-identified by emby, and as a fellow TMM user, you will understand why I used TMM to try to correct it.  After that, I re-ran a scan in emby, and it picked up the changes no problem.  

 

On the other hand, we KNOW that TMM will NOT pick up the changes, so any changes that you make in emby will have to be done manually in TMM,

[thwaller 2]

Yeah, it's a potential minefield.  I believe that I have proved to myself how much influence each has over the content of the nfo files, though.  I am not positive that I can explain it, but will try.  

 

I found a tv series which was mis-identified by emby, and as a fellow TMM user, you will understand why I used TMM to try to correct it.  After that, I re-ran a scan in emby, and it picked up the changes no problem.  

 

On the other hand, we KNOW that TMM will NOT pick up the changes, so any changes that you make in emby will have to be done manually in TMM,

 

Thanks for sharing. I see it the same way, I just have not seen it for myself, but that is ok. Essentially, I would need to either turn off Emby's ability to write to NFO, or just consider the changes that Emby might make as temporary..... thinking as a general rule. Can I ask what other systems you have tried? I have used Kodi in the past, quite extensively, so I feel I am quite aware of what it can do, how it works, etc. I tried Plex for a short time (ike a day) but did not look into it too much before trying Emby. That is my media manager experience. Basically looking at Plex vs Emby, or other or none. I can access all my media via DLNA and can use Chromecast to do most all I need, but I would something to allow me to browser my media content graphically, which Emby, Plex, Kodi, etc all provide. I want something that will play and use my media, not something that wants to take over anything or take control of anything. But as such a feature would likely be there for users that do not manage their media separately, also have the ability to turn it off and control its automated actions.

 

I believe that my choice is down to Plex or Emby from what I have found and tested. My only issue is I am unsure which is better suited. Neither really meet my desires, mostly because of lack of control. Things like automated tasks, scans, syncs, etc. Do you have any experience to share, and why you chose to use Emby over other options? During my time without such a system, I have been using VLC a lot, and also Archos media player for its casting ability, although it is somewhat buggy.

[alexpaton 10]

I almost exclusively used Kodi for my media, with Trakt to keep track of my watched history.  I use multiple devices, so this isn't ideal.  Also, for some reason, my FireTV box 'loses' its libraries, usually when it goes to sleep.  My first step was to share my Kodi library, which was a disaster.  It didn't work at all with one of my android boxes, and the FireTV once again lost the libraries.

 

That led me to TMM, which at least allowed me to re-add the libraries, much more quickly and with less scraping.  The FireTV is a real pain for scraping, because it has sleep settings that can't be altered, and it has a habit of going to sleep in the middle (or 5 minutes in).

 

I actually found emby while searching for plex, which I tried out ages ago.  I tried Plex first, which was the biggest disaster of all.  It's vaguely possible that the TMM nfo files were the cause of it, but I'm not sure.  Basically, importing 1 folder of movies, led to a 12GB log file and about 8GB of movie images, which filled my root partition to the point that the whole system stopped working properly.  It took me ages to get the system working properly.

 

The things that most attract me to Emby over Plex (apart from the above) is the way that it handles mixed content, and IPTV.  Even though the IPTV on Plex is much more finished than in Emby, it seems to work so much better in Emby.  There are also more options for backend tuner connectivity & varied xmltv schedules for EPG.

 

As to the Mixed Content element, I am over the moon about that.  Kodi has 2 ways of dealing with stuff it can't scrape:-

1) ignore it. You either have to search through the logs, or run an addon like Missing Movies, which doesn't actually get you much closer to them being in your library.

2) Display it as a different movie/series.

 

For things like sports, Kodi has no way of dealing with it at all.  If I search for my favourite football (soccer) team, and Emby presents me with a list of matches, AND it includes anything on the Tv Guide, AND any matches that I have recorded.  All of that gets (mostly) sent over to Kodi as well.

 

It is unlikely that Emby will ever be perfect, simply because we all have different tastes, BUT it is by far the nearest I have found.  One last thing: the developers on here are soooooo responsive, sometimes fixing bugs and adding functionality with a day.

[thwaller 2]

Thank you for the information. Although this changes the initial topic of this post, it does carry on a conversation.... I am experiencing more issues with Emby maybe you can help with? I have a large library that is in pieces. Two primary pieces and two smaller pieces. The two smaller pieces are more temporary libraries and not an issue here. The two main pieces however are. One is a NAS and the other a USB storage device. These devices are not always present ... sometimes I have only temporary libraries and sometimes the NAS and/or USB device are present. The issue I am having is that if a device is missing for too long, the library disappears from Emby and I need to rescan it in. This is no big deal for a small library, but this takes me a long time to do. I am not timing it, but it has been over an hour and I am still at 86%. It will finish, but it takes a long time. Now, tMM detects the absence of a missing source and acts accordingly, but it appears that Emby is seeing a library as missing and just removes it. This makes sense for pieces of a library, but when the whole thing is missing, there seems to be an alternative reason besides complete deletion. I thought I saw a feature, although that could have been in Plex, to change this in settings, but I cannot find it now. It seems that all of the scheduled tasks are fixed and will run whether you want them to or not. Is this true or am I missing where such things can be changed?

 

There are smaller issues I am having that I am fine with, like these errors already reported on failure to update plugins. I will deal with such things because, as you, I am finding Emby to be "the nearest I have found". Unfortunately, the issues above are a deal breaker for me. I cannot spend hours rebuilding libraries each time I wish to watch a movie or show. This is one thing I did like about Kodi ... you could configure it to preform as you want or need. Very little was forced on you and even when there was no GUI settings, there were text based files you could edit to modify behaviour if you were willing to dig through everything. I find that Plex and Emby both have an issue of restricting the user too much, at least as it appears to me. IF you could help in any way I would appreciate that. This does appear to be the lightest and cleanest product I have tried, which is a big thing for me.

[mastrmind11 717]

 

 

The issue I am having is that if a device is missing for too long, the library disappears from Emby and I need to rescan it in

This was supposed to have been fixed many months ago.  @@Luke is there a time constraint hard coded before Emby removes a missing library?  I'd expect them to stay indefinitely, until the user decides to remove it manually.

[Luke 37251]

It shouldn't be disappearing at all anymore, assuming the folder that is missing represents the top of the library added to emby server. it should stay there and simply fail to scan. If it's a sub-folder or if there are sym links involved then it could be a little harder for the server to differentiate between offline/failed vs deleted/non-existant, and that could explain that. but it's better that we look at a specific example.

[thwaller 2]

It is a folder "movies". It contains folders named ,"#", "A", "B", etc. Each of those folders contains one folder per movie, and those folders contain the movie itself with the nfo and artwork, etc. The are no links, hard soft or otherwise, and there is no variation in the folder/file structure. The structure is repetitive on any media source used.

 

Example for data source "<NAS> /movies/":

<NAS> /movies/

<NAS> /movies/A/

<NAS> /movies/B/

<NAS> /movies/C/

<NAS> /movies/C/Condemned/

<NAS> /movies/C/Conan/

<NAS> /movies/C/Conan/Conan.mp4

<NAS> /movies/C/Conan/Conan.nfo

<NAS> /movies/C/Conan/Conan.ass

 

When the data source is absent, <NAS> is not there, thus the /movies/ share is not there either. I often use my "player", what ever it might be, with and/or without media sources. I would expect (although not a requirement) that the entities are all cached to a small degree so they still appear, but they will not play as there is no source. I have each source as its own library, so if one source is gone, it means that one of the libraries is gone in its entirety. So if I have 3 data sources, I have three libraries. there is no intermixing.

Edited February 1, 2019 by thwaller

[Luke 37251]

Ok, I'd have to see the server log from when they were removed. if you could recreate the situation that would be helpful. thanks.

[thwaller 2]

Ok, I'd have to see the server log from when they were removed. if you could recreate the situation that would be helpful. thanks.

 

Well, there is a history of log files, but I have no clue what I am looking for. Could you direct me to a flag or something I can do a text search for? I can get you a log file straight away. If there is something more that is not being said, please PM me.

[Luke 37251]

if an entire library is being deleted then there would be a huge amount of entries saying 

Deleting item, Type:

[thwaller 2]

Ok. I have found such an entry in the existing log files. This is contained in a 18MB log file. I would assume you need only the portion from start up to the following? To where should this be sent?

2019-02-01 04:58:06.781 Info App: Deleting item, Type: Folder, Name: E, Path: /media/movie/E, Id: 3050

2019-02-01 04:58:08.963 Info App: Deleting item, Type: Folder, Name: Y, Path: /media/movie/Y, Id: 3051

2019-02-01 04:58:09.279 Info App: Deleting item, Type: Folder, Name: S, Path: /media/movie/S, Id: 3052

2019-02-01 04:58:15.738 Info App: Deleting item, Type: Folder, Name: X, Path: /media/movie/X, Id: 3053

2019-02-01 04:58:15.955 Info App: Deleting item, Type: Folder, Name: K, Path: /media/movie/K, Id: 3054

2019-02-01 04:58:17.344 Info App: Deleting item, Type: Folder, Name: V, Path: /media/movie/V, Id: 3055

2019-02-01 04:58:18.064 Info App: Deleting item, Type: Folder, Name: P, Path: /media/movie/P, Id: 3057

2019-02-01 04:58:20.547 Info App: Deleting item, Type: Folder, Name: R, Path: /media/movie/R, Id: 3058

2019-02-01 04:58:22.671 Info App: Deleting item, Type: Folder, Name: G, Path: /media/movie/G, Id: 3060

2019-02-01 04:58:24.843 Info App: Deleting item, Type: Folder, Name: U, Path: /media/movie/U, Id: 3061

2019-02-01 04:58:25.694 Info App: Deleting item, Type: Folder, Name: A, Path: /media/movie/A, Id: 3062

2019-02-01 04:58:28.725 Info App: Deleting item, Type: Folder, Name: C, Path: /media/movie/C, Id: 3064

2019-02-01 04:58:32.033 Info App: Deleting item, Type: Folder, Name: I, Path: /media/movie/I, Id: 3065

2019-02-01 04:58:33.732 Info App: Deleting item, Type: Folder, Name: Z, Path: /media/movie/Z, Id: 3066

2019-02-01 04:58:33.882 Info App: Deleting item, Type: Folder, Name: J, Path: /media/movie/J, Id: 3068

2019-02-01 04:58:34.665 Info App: Deleting item, Type: Folder, Name: B, Path: /media/movie/B, Id: 3069

2019-02-01 04:58:38.396 Info App: Deleting item, Type: Folder, Name: Q, Path: /media/movie/Q, Id: 3070

2019-02-01 04:58:38.466 Info App: Deleting item, Type: Folder, Name: W, Path: /media/movie/W, Id: 3071

2019-02-01 04:58:40.685 Info App: Deleting item, Type: Folder, Name: F, Path: /media/movie/F, Id: 3072

2019-02-01 04:58:42.415 Info App: Deleting item, Type: Folder, Name: L, Path: /media/movie/L, Id: 3073

2019-02-01 04:58:44.361 Info App: Deleting item, Type: Folder, Name: D, Path: /media/movie/D, Id: 3074

2019-02-01 04:58:46.523 Info App: Deleting item, Type: Folder, Name: M, Path: /media/movie/M, Id: 3077

2019-02-01 04:58:48.896 Info App: Deleting item, Type: Folder, Name: T, Path: /media/movie/T, Id: 3079

2019-02-01 04:58:50.480 Info App: Deleting item, Type: Folder, Name: H, Path: /media/movie/H, Id: 3080

2019-02-01 04:58:52.437 Info App: Deleting item, Type: Folder, Name: N, Path: /media/movie/N, Id: 3082

2019-02-01 04:58:53.236 Info App: Deleting item, Type: Folder, Name: O, Path: /media/movie/O, Id: 3083

2019-02-01 04:58:54.018 Info App: Deleting item, Type: Folder, Name: Æ, Path: /media/movie/Æ, Id: 66335

2019-02-01 04:58:54.039 Info App: Deleting item, Type: Folder, Name: #, Path: /media/movie/#, Id: 66336

[Luke 37251]

The entire log would be ideal. You can attach it here. Thanks.

[thwaller 2]

Given the nature of the log, I will not post it publicly. I will share it with you directly, you meaning you yourself or whom ever needs to see it. This should have been an obvious response as the log contains forms of linked PII.

[Luke 37251]

Yes you can always PM it to me, thanks.