Jump to content

fast sync not so fast with lots of updates that shouldn't be included


shred00

Recommended Posts

Angelblue05

Yes, I'll get you that information. Thanks Luke! If we can get the server working less, it's always a good thing in my book :)

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

Angelblue05

@@Luke

 

I've look deeper into my logs, never mind what I said about virtual items, I think I corrected this a while back. In my case, it seems the same items that keep being updated are ones emby are throwing errors about. I'll get back to you on this.

 

@@shred00

 

Do provide your server log so Luke can take a look. Do you see those same lines every day?

Edited by Angelblue05
Link to comment
Share on other sites

sualfred

@@Luke

I'm also facing it daily or after a library scan for 104 items.

I'll send you a PM with all your requested details.

Link to comment
Share on other sites

llonca13

I also have to report the same problem happening and its very frustrating. The problem started to appear 1-3weeks ago I cant tell for sure,maybe after a new update (i run it in Docker, on auto update so I cant be more precise when).

 

Incremental sync doesnt happen, instead full scan is started (like when you go to emby addon and hit manual sync option). I thought its problem on my end so I performed database reset (and even reinstalled emby addon and deleted emby.db and myvideos.db) to start things fresh, and even reinstalled my emby server (deleted config, start from scratch). The problem continues, no incremental updates.

 

Im using emby (with fast start plugin) on android spmc (kodi 16.1,mibox), LE (rpi3 17.5,generic s905x and odroid c2 on 17.6) and all devices experience the same bug (even after reinstall).

I have correct season structure and this problems started to arise since a few weeks ago.

 

Im on newest stable emby server, stable emby kodi addon, and version 2.0 of fast sync kodi plugin. Running in native paths mode.

 

Please tell me what logs you need to identify this bug.

Edited by llonca13
Link to comment
Share on other sites

sualfred

@Illonca13

 

Turn on Debug logging in the server dashboard and restart the server. After it's back up again trigger a manual library scan. As soon it's done upload the server log.

Link to comment
Share on other sites

sualfred

@@Luke

 

I guess I've found the issue. The .nfo updating on each library scans happens as soon as .sub (idx + sub file) subtitles are stored next to the video. It does not happen if .srt subs are present.

 

@@llonca13 @@shred00

Could you please check it on your side, too?

 

-> Move the subtitle files from one of the unnecessary updated library items to a temporary place

-> Scan library (to remove the existing subtitles informations from the nfo file)

-> Wait a minute

-> Scan library again and check if the modification date of the video .nfo has been changed again

Edited by sualfred
Link to comment
Share on other sites

sualfred

@@shred00

 

Can you please provide a full debug log so I can compare if we both facing the similar issues? Because for me it's def. caused by sub/idx subtitles (for whatever reason)

Link to comment
Share on other sites

sualfred

I don't know. It changes anything.
 


      <subtitle>
        <codec>subrip</codec>
        <micodec>subrip</micodec>
        <language>ger</language>
        <scantype>progressive</scantype>
        <default>True</default>
        <forced>True</forced>
      </subtitle>
      <subtitle>
        <codec>sub</codec>
        <micodec>sub</micodec>
        <language>deu</language>
        <scantype>progressive</scantype>
        <default>False</default>
        <forced>True</forced>
      </subtitle>
      <subtitle>
        <codec>sub</codec>
        <micodec>sub</micodec>
        <language>eng</language>
        <scantype>progressive</scantype>
        <default>False</default>
        <forced>False</forced>
      </subtitle>
      <subtitle>
        <codec>sub</codec>
        <micodec>sub</micodec>
        <language>deu</language>
        <scantype>progressive</scantype>
        <default>False</default>
        <forced>False</forced>
      </subtitle>

The subs are recongnized and written correctly, but I assume Emby thinks that the information for the SUB subtitles is missing in the .nfo and rewrites it again with the same content. That leads into a changed .nfo modification date and a log entry with ...
 

2018-01-11 10:41:35.453 Debug App: Running FFProbeProvider for /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv
2018-01-11 10:41:35.453 Debug MediaEncoder: /opt/emby-server/bin/ffprobe -i file:"/mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format
2018-01-11 10:41:35.472 Debug App: Running MovieNfoProvider for /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv
2018-01-11 10:41:35.473 Debug App: Running OmdbItemProvider for /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv
2018-01-11 10:41:35.474 Debug App: Running MovieDbProvider for /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv
2018-01-11 10:41:35.474 Debug JsonSerializer: Deserializing file /var/lib/emby/cache/tmdb-movies2/68726/all-de.json
2018-01-11 10:41:35.477 Debug App: Running BaseGenreCleaner for /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv
2018-01-11 10:41:35.486 Debug ProviderManager: Saving /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv to Nfo.
2018-01-11 10:41:35.495 Debug App: Saving /mnt/omv1/omv1/filme/Pacific Rim (2013)/Pacific Rim (2013) h264-1080p DTS-6ch.mkv to database.
2018-01-11 10:41:35.496 Debug App: Emby.Kodi.SyncQueue: ItemUpdated added for DB Saving 5d9e44c3-5c6d-3755-a9fe-47e17bfb5afa

... and voila, the SyncQueue addon is triggered on a library scan.

 

Still trying to find out why this is happening, because I have hundreds of .sub subtitles in my database, but it only applies for ~104 items.

  • Like 1
Link to comment
Share on other sites

Angelblue05

In the Kodi log, you should be able to identify the title of one of the 104 items. Best thing to do is track the nfo changes and subs. Copy paste into notepad++ and compare to after a scan happens to see if anything changes?

 

Anyway, hope @@Luke will know why it's happening.

 

Sent from my iPhone using Tapatalk

Edited by Angelblue05
Link to comment
Share on other sites

sualfred

I already did. It happens to the ones with the .sub issue from above. And the .nfos are the same and just gets rewritten to the file. As soon as I delete the .sub subtitles the issue is gone.

  • Like 1
Link to comment
Share on other sites

sualfred

I'm currently cleaning all my directories which are affected and I've noticed that this also happens for some videos with .srt subtitles.

@@Luke
Please let me know if I should upload a sample video (complete directory), if you cannot reproduce it.

  • Like 1
Link to comment
Share on other sites

llonca13

Im sorry im not home now I will upload tomorrow.

 

But I can assure you I dont have any .sub subtitle in any of my directorys. Every single media file in my media library have .srt subtitle with the same name as my file (downloaded through kodi).

Edited by llonca13
Link to comment
Share on other sites

shred00

@@shred00

 

Can you please provide a full debug log so I can compare if we both facing the similar issues? Because for me it's def. caused by sub/idx subtitles (for whatever reason)

 

I mentioned previously that I don't do anything at all with subtitles so my logs will not help in that respect.

 

I did PM @@Luke with the logs he was requesting but I guess he has not had time to look at them yet.  I am sure he will when he has time.

 

In the meanwhile I have converted all of my episodes into Season folders to see if my issue is with virtual seasons as suspected.  I will report when I have a result of that.

Edited by shred00
  • Like 1
Link to comment
Share on other sites

shred00

Too soon to determine yet.  I need some time to pass between doing that (and the manual resync that I did after doing that, to get a clean slate to work from) to see how the incremental sync acts.  I should know in the next day or 3.

  • Like 1
Link to comment
Share on other sites

shred00

So, the problem is not virtual season folders.

 

I have all of my episodes in real Season NN folders and I still had "fast sync"s last night wanting to update over 500 items and taking minutes to do it.

 

I'm also noticing that even though I have actual Season NN folder dirs in my library now, I am still getting over a thousand:

2018-01-18 10:41:27.445 Info App: Creating Season Season 2015 entry for Saturday Night Live
2018-01-18 10:41:27.720 Info App: Creating Season Season 2016 entry for Saturday Night Live
2018-01-18 10:41:28.014 Info App: Creating Season Season 2017 entry for Saturday Night Live
2018-01-18 10:41:28.479 Info App: Creating Season Season 2014 entry for The Mentalist
2018-01-18 10:41:28.553 Info App: Creating Season Season 2015 entry for The Mentalist
2018-01-18 10:41:28.751 Info App: Creating Season Season 2017 entry for Marketplace
2018-01-18 10:41:29.014 Info App: Creating Season Season 2016 entry for X Company

type messages in a 24 hour period.  I understood those were virtual seasons being created.  Why are virtual seasons still being created here when I have all of my episodes in actual season folders (where some of those seasons are the year, not a season number)?

 

So what next?  Were the log files I PM'd to @@Luke previously any use?

Link to comment
Share on other sites

Angelblue05

Wow thanks for the info. @@Luke

That's definitely not normal, what are those log lines then?

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

shred00

Can you try the new release and see if that makes any difference? thanks.

Have upgraded to 3.2.70.0+66.1.  Will let you know over the next few days if fast syncs are still large with many updates for things that really shouldn't have any change to them.

Link to comment
Share on other sites

shred00

@@shred00

 

Any changes?

 

Ha!  I was just coming here to provide an update when I saw the above message.

 

So, unfortunately not.  The last few times I have logged into an account that logs in pretty much daily there have been over 500 updates included in the fast sync.

Link to comment
Share on other sites

shred00

Definitely not as many as I was, but yes, still seeing them:

$ grep Creating\ Season /var/lib/emby-server/logs/server-63652348800.txt | wc -l
128
$ grep Creating\ Season /var/lib/emby-server/logs/server-63652348800.txt | head -23
2018-01-24 04:03:16.741 Info App: Creating Season Season 2016 entry for The Simpsons
2018-01-24 04:03:17.794 Info App: Creating Season Season 2016 entry for The Simpsons
2018-01-24 04:03:39.615 Info App: Creating Season Season 2016 entry for Scandal (2012)
2018-01-24 04:03:40.205 Info App: Creating Season Season 2016 entry for Scandal (2012)
2018-01-24 04:04:52.828 Info App: Creating Season Season 2016 entry for Vikings
2018-01-24 04:04:53.332 Info App: Creating Season Season 2016 entry for Vikings
2018-01-24 10:44:46.138 Info App: Creating Season Season 2014 entry for The Simpsons
2018-01-24 10:44:46.317 Info App: Creating Season Season 2016 entry for The Simpsons
2018-01-24 10:45:00.443 Info App: Creating Season Season 2016 entry for The Simpsons
2018-01-24 10:45:01.232 Info App: Creating Season Season 2013 entry for Cougar Town
2018-01-24 10:45:02.750 Info App: Creating Season Season 2017 entry for The American Woodshop
2018-01-24 10:45:03.859 Info App: Creating Season Season 2016 entry for X Company
2018-01-24 10:45:04.410 Info App: Creating Season Season 2014 entry for Holmes on Homes
2018-01-24 10:45:32.045 Info App: Creating Season Season 2013 entry for Woodsmith Shop
2018-01-24 10:45:32.751 Info App: Creating Season Season 2016 entry for Woodsmith Shop
2018-01-24 10:45:32.905 Info App: Creating Season Season 2017 entry for Woodsmith Shop
2018-01-24 10:45:33.567 Info App: Creating Season Season 2016 entry for Rough Cut - Woodworking with Tommy Mac
2018-01-24 10:45:34.116 Info App: Creating Season Season 2017 entry for Rough Cut - Woodworking with Tommy Mac
2018-01-24 10:45:34.568 Info App: Creating Season Season 2017 entry for The West BLock
2018-01-24 10:45:35.029 Info App: Creating Season Season 2018 entry for The West BLock
2018-01-24 10:45:38.921 Info App: Creating Season Season 2015 entry for Marketplace
2018-01-24 10:45:38.994 Info App: Creating Season Season 2016 entry for Marketplace
2018-01-24 10:45:39.831 Info App: Creating Season Season 2017 entry for Marketplace
  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...