Emby for Kodi stops syncing once a month like clock work

[quickmic 1477]

Just now, brian1709 said:

Hi, not sure. I usually add a whole bunch of stuff once or twice a week. Some new shows, some new seasons and just episodes.

What I did was delete Emby Next Gen, as well as the DB and then re installed. It ran through everything perfectly this time.

Of course now we'll have to see when I add new stuff. I'll keep you updated.

Thanks, I try to design a test case. The issue is quite clear to me, but not sure what's the cause.

[quickmic 1477]

I think I found the problem. I'll release a new version soon...

[quickmic 1477]

Please try this version:

https://kodi.emby.tv/Public testing/plugin.video.emby-next-gen-6.0.4-build 9 (ex2).zip

[brian1709 9]

will do. Just a quick question. In the older versions of Emby for Kodi there was options to sync manually or with EmbyCon. How does it sync now?

Well not really how but what is triggering a sync other than a start of Kodi? is there a constant check for DB changes or a specific time frame?

I remember PLEX had an issue where it was constantly syncing or scanning that used an insane amount of resources. While it is not an issue on my test machine, some of the lower end kodi boxes in the house only have ARM processors and 2 Gb RAM. So is there a way to switch to manual sync or change the way it sync?

 

Edited August 13, 2021 by brian1709

[brian1709 9]

As an update, I added four files before I received 6.0.4 v.9 and they all synced fine, but I will update to v.9

[quickmic 1477]

11 minutes ago, brian1709 said:

 In the older versions of Emby for Kodi there was options to sync manually or with EmbyCon.

 

I'm not aware of such an option in 4.x but I never used EmbyCon and Emby for Kodi at the same time.

Quote

How does it sync now?

It (incremental) syncs on each Kodi start and also in realtime via Websocket communication with emby server. It's completely independent from EmbyCon.

 

Quote

insane amount of resources.

Memory issues are solved, next-gen is almost completely rewritten. 4.x for emby and the Plex kodi plugin are very similar and both have major design flaws.

 

 

Edited August 13, 2021 by quickmic

[brian1709 9]

Ok, stopped syncing again after I added some files. Could it be due to folder structure? (I had not yet updated to 4.0.6 v9)

It seems that if I add just a media file into an existing folder structure it can deal with it, but when I add a new folder into an existing show it freezes

ie.  <Show Name>  

            <Season Number>

                  SxxExx

See log1

I updated to 4.0.6 v. 9 no change still does exactly nothing, not even start and freeze.

It logs into Emby when Kodi starts but then just does nothing. Restarting Emby from the settings menu in Emby does exactly the same.

See log2

 

Looking at the log the thing that strikes me is that it gives an error on finding a server but then authenticate the user ??

 

2021-08-14 13:23:56.495 T:11664    INFO <general>: INFO: EMBY.emby.main: ---[ INIT EMBYCLIENT: ]---
2021-08-14 13:23:56.495 T:11664    INFO <general>: INFO: EMBY.emby.main: --[ server/DEFAULT ]
2021-08-14 13:23:56.495 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: Begin connect
2021-08-14 13:23:57.497 T:11664   ERROR <general>: ERROR: EMBY.core.connection_manager: Error trying to find servers: timed out
2021-08-14 13:23:57.497 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: Begin connectToServers, with 1 servers
2021-08-14 13:23:57.497 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: testing connection mode 0 with server HTPC
2021-08-14 13:23:57.497 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: tryConnect url: http://10.20.30.178:8096/emby/system/info/public
2021-08-14 13:23:57.502 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: calling onSuccessfulConnection with connection mode 0 with server HTPC
2021-08-14 13:23:57.516 T:11664    INFO <general>: INFO: EMBY.emby.main: User is authenticated.
2021-08-14 13:23:57.617 T:11664    INFO <general>: INFO: EMBY.emby.main: ---[ START EMBYCLIENT: b8424b13a59f42d6b4df3f0be4e65ad3 ]---
2021-08-14 13:23:57.632 T:11664 WARNING <general>: WARNING: EMBY.library.Library: --->[ library ]
2021-08-14 13:23:57.638 T:13428    INFO <general>: INFO: Emby.hooks.websocket: --->[ websocket ]
2021-08-14 13:23:57.638 T:11664    INFO <general>: INFO: EMBY.emby.main: [ Server Online ]

 

kodi1.log kodi2.log

[brian1709 9]

I am going to delete DB again and re-install new version of Emby Next Gen without changing folder structure to see if takes it.

Needless to say EmbyCon is perfect

[brian1709 9]

deleted DB and re-installed emby NG. Ran perfectly with all the folders and files that it froze on last time.

[quickmic 1477]

On 8/14/2021 at 1:47 PM, brian1709 said:

Ok, stopped syncing again after I added some files. Could it be due to folder structure? (I had not yet updated to 4.0.6 v9)

It seems that if I add just a media file into an existing folder structure it can deal with it, but when I add a new folder into an existing show it freezes

ie.  <Show Name>  

            <Season Number>

                  SxxExx

See log1

I updated to 4.0.6 v. 9 no change still does exactly nothing, not even start and freeze.

It logs into Emby when Kodi starts but then just does nothing. Restarting Emby from the settings menu in Emby does exactly the same.

See log2

 

Looking at the log the thing that strikes me is that it gives an error on finding a server but then authenticate the user ??

 

2021-08-14 13:23:56.495 T:11664    INFO <general>: INFO: EMBY.emby.main: ---[ INIT EMBYCLIENT: ]---
2021-08-14 13:23:56.495 T:11664    INFO <general>: INFO: EMBY.emby.main: --[ server/DEFAULT ]
2021-08-14 13:23:56.495 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: Begin connect
2021-08-14 13:23:57.497 T:11664   ERROR <general>: ERROR: EMBY.core.connection_manager: Error trying to find servers: timed out
2021-08-14 13:23:57.497 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: Begin connectToServers, with 1 servers
2021-08-14 13:23:57.497 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: testing connection mode 0 with server HTPC
2021-08-14 13:23:57.497 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: tryConnect url: http://10.20.30.178:8096/emby/system/info/public
2021-08-14 13:23:57.502 T:11664    INFO <general>: INFO: EMBY.core.connection_manager: calling onSuccessfulConnection with connection mode 0 with server HTPC
2021-08-14 13:23:57.516 T:11664    INFO <general>: INFO: EMBY.emby.main: User is authenticated.
2021-08-14 13:23:57.617 T:11664    INFO <general>: INFO: EMBY.emby.main: ---[ START EMBYCLIENT: b8424b13a59f42d6b4df3f0be4e65ad3 ]---
2021-08-14 13:23:57.632 T:11664 WARNING <general>: WARNING: EMBY.library.Library: --->[ library ]
2021-08-14 13:23:57.638 T:13428    INFO <general>: INFO: Emby.hooks.websocket: --->[ websocket ]
2021-08-14 13:23:57.638 T:11664    INFO <general>: INFO: EMBY.emby.main: [ Server Online ]

 

kodi1.log 104.6 kB · 0 downloads kodi2.log 141.16 kB · 0 downloads

Sorry for the delay, I was busy on weekend.

It's not a critical issue, seems your server response was too late. In such an event next-gen tries to reconnect and obviously works after the first attempt. I'll check it anyway. Also you are affected from the bug in the < 6.0.4 v9 bug but should be solved after update.

Edited August 16, 2021 by quickmic

[brian1709 9]

It stopped again. No sync, no DB update. When I start Kodi it logs into Emby Server and then just do nothing. A message briefly appears that say something like .." updating database changes..' but then immediately vanishes and nothing happens.

All new added media files appears perfectly in EmbyCon and all played media changes to the correct information in Kodi menus like next up or progress. But it will not add new media.

I tried restarting Emby next gen from within the addon itself same result nothing.

'Repair the DB' from wthin Emby next gen addon just deleted the whole DB. This is just no good

kodi.log

Edited August 20, 2021 by brian1709
left out some info

[brian1709 9]

Also Emby Next Gen has no manage Libraries option anymore. There is some functions under  <settings>  <sync> <manage libraries>

but no <update libraries> option. none of the options available does anything. <add libraries> only allows for libraries that has been newly created.

<Repair libraries>  deletes everything and builds a new one which takes forever

Also under the sync menu there is a statement at the bottom...'Update or repair your libraries to apply the changes below'  but there is no 'below' or 'update' option

The only option in Emby Next Gen that does anything is <settings> <Emby for Kodi> <restart Emby for Kodi> which restarts Emby but still does not sync new media to the Kodi DB although it is in EmbyCon.

[brian1709 9]

of course deleting the DBs in <Kodi> <UserData> rebuilds the DB perfectly but I cannot be doing that every 4-5 days and also should not have to.

Why does it work for a while and then just stop. Everything that was added was just episodes to existing shows. No new shows or folders.

[quickmic 1477]

8 hours ago, brian1709 said:

It stopped again. No sync, no DB update. When I start Kodi it logs into Emby Server and then just do nothing. A message briefly appears that say something like .." updating database changes..' but then immediately vanishes and nothing happens.

All new added media files appears perfectly in EmbyCon and all played media changes to the correct information in Kodi menus like next up or progress. But it will not add new media.

I tried restarting Emby next gen from within the addon itself same result nothing.

'Repair the DB' from wthin Emby next gen addon just deleted the whole DB. This is just no good

kodi.log 5.18 MB · 0 downloads

There is a bug reported in your log. Sounds like a timing issue, I'll fix it soon.

[quickmic 1477]

1 hour ago, brian1709 said:

of course deleting the DBs in <Kodi> <UserData> rebuilds the DB perfectly but I cannot be doing that every 4-5 days and also should not have to.

Why does it work for a while and then just stop. Everything that was added was just episodes to existing shows. No new shows or folders.

I'll check that.

[quickmic 1477]

1 hour ago, brian1709 said:

update libraries

Indeed, update libraries was removed. Actually this option was just a workaround for invalid syncs as you experiencing.

I should never be necessary to update manually, my intention is to fix the core of a issue not using workarounds.

[quickmic 1477]

@brian1709

Currently I'm performing tests for new shows. All good so far, let me ask some questions...

Is the websocket connection working? You can test this by sending a message from Emby server to Kodi client.

When did you add a TVShows? While Kodi was running or Kodi stopped?

If running, try Kodi restart.

fyi, Kodi syncs content completely different while startup compared with runtime updates.

Runtime syncs uses the websocket connection (that's why you should test it). If websocket is not working, no updates will be pushed from Emby server BUT. It should sync on Kodi start. The plugin actively queries Emby sync data on Kodi startup and not waiting for data pushed from Emby server.

 

The issue in the log came from Exit before repair sync was completed, not critical but I'll fix it anyway.

Also I can see in the log you watched a video while you triggered the repair sync. Syncs will paused while videos are playing. I'll check the repair sync code, currently I assume:

The delete content job is not paused while video playback. Therefore all the content was deleted (so far correct), next job: Updates are started but due to playback is was immediately paused.

According to the log, you never stopped the playback until Kodi exit:

 

2021-08-19 23:05:10.533 T:3568     INFO <general>: INFO: EMBY.hooks.player.Player: [ onPlayBackStopped ]
2021-08-19 23:05:10.534 T:3568     INFO <general>: INFO: EMBY.hooks.player.Player: --<[ playback ]
2021-08-19 23:05:10.536 T:3568  WARNING <general>: WARNING: EMBY.entrypoint.Service: ---<[ EXITING ]
2021-08-19 23:05:10.543 T:10292    INFO <general>: INFO: EMBY.hooks.webservice.WebService: ---<[ webservice/57578 ]
2021-08-19 23:05:10.551 T:3568     INFO <general>: INFO: EMBY.emby.main: ---[ STOP EMBYCLIENT: b8424b13a59f42d6b4df3f0be4e65ad3 ]---
2021-08-19 23:05:10.551 T:3568  WARNING <general>: WARNING: EMBY.core.HTTP: --<[ session/2206599464800 ]
2021-08-19 23:05:10.552 T:1524     INFO <general>: INFO: Emby.hooks.websocket: ---<[ websocket ]

The last logs before Kodi was shutdown came from sync queue. It was idle waiting for playback stop. That never happened until you exit Kodi.

 

Quote

2021-08-19 23:05:10.559 T:12964    INFO <general>: INFO: EMBY.library.Library: -->[ worker_update started ]
2021-08-19 23:05:10.559 T:12964    INFO <general>: INFO: EMBY.library.Library: [ worker_update queue size ] 73311
2021-08-19 23:05:10.560 T:12964    INFO <general>: INFO: EMBY.library.Library: [ worker_update paused ]
2021-08-19 23:05:10.560 T:12888    INFO <general>: INFO: EMBY.library.Library: -->[ worker_userdata started ]
2021-08-19 23:05:10.560 T:12888    INFO <general>: INFO: EMBY.library.Library: [ worker_userdata queue size ] 0
2021-08-19 23:05:10.561 T:12888    INFO <general>: INFO: EMBY.library.Library: -->[ worker_update started ]
2021-08-19 23:05:10.561 T:12888    INFO <general>: INFO: EMBY.library.Library: [ worker_update queue size ] 73311
2021-08-19 23:05:10.561 T:12888    INFO <general>: INFO: EMBY.library.Library: [ worker_update paused ]
2021-08-19 23:05:10.561 T:12888    INFO <general>: INFO: EMBY.library.Library: -->[ worker_remove started ]
2021-08-19 23:05:10.561 T:12888    INFO <general>: INFO: EMBY.library.Library: [ worker_remove queue size ] 0

The following error was just a side effect. Remove queue was starting the job but Kodi shutdown was already in progress and some modules are already unloaded (object sync)...

 

Quote

2021-08-19 23:05:10.958 T:6388    ERROR <general>: Thread-47
2021-08-19 23:05:10.958 T:6388    ERROR <general>: :
                                                   
2021-08-19 23:05:10.958 T:6388    ERROR <general>: Traceback (most recent call last):
                                                   
2021-08-19 23:05:10.958 T:6388    ERROR <general>:   File "C:\Program Files\Kodi\system\python\Lib\threading.py", line 932, in _bootstrap_inner
                                                   
2021-08-19 23:05:10.959 T:6388    ERROR <general>:     
2021-08-19 23:05:10.959 T:6388    ERROR <general>: self.run()
2021-08-19 23:05:10.959 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.959 T:6388    ERROR <general>:   File "C:\Program Files\Kodi\system\python\Lib\threading.py", line 870, in run
                                                   
2021-08-19 23:05:10.965 T:6388    ERROR <general>:     
2021-08-19 23:05:10.965 T:6388    ERROR <general>: self._target(*self._args, **self._kwargs)
2021-08-19 23:05:10.965 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.965 T:6388    ERROR <general>:   File "C:\Users\Brian\AppData\Roaming\Kodi\addons\plugin.video.emby-next-gen\helper\pluginmenu.py", line 95, in select_managelibs
                                                   
2021-08-19 23:05:10.966 T:6388    ERROR <general>:     
2021-08-19 23:05:10.966 T:6388    ERROR <general>: self.manage_libraries(0)
2021-08-19 23:05:10.966 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.966 T:6388    ERROR <general>:   File "C:\Users\Brian\AppData\Roaming\Kodi\addons\plugin.video.emby-next-gen\helper\pluginmenu.py", line 108, in manage_libraries
                                                   
2021-08-19 23:05:10.966 T:6388    ERROR <general>:     
2021-08-19 23:05:10.966 T:6388    ERROR <general>: self.EmbyServers[EmbyServerId].library.select_libraries("RepairLibrarySelection")
2021-08-19 23:05:10.966 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.966 T:6388    ERROR <general>:   File "C:\Users\Brian\AppData\Roaming\Kodi\addons\plugin.video.emby-next-gen\database\library.py", line 433, in select_libraries
                                                   
2021-08-19 23:05:10.967 T:6388    ERROR <general>:     
2021-08-19 23:05:10.967 T:6388    ERROR <general>: self.remove_library(libraries[x - 1]['Id'])
2021-08-19 23:05:10.967 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.967 T:6388    ERROR <general>:   File "C:\Users\Brian\AppData\Roaming\Kodi\addons\plugin.video.emby-next-gen\database\library.py", line 539, in remove_library
                                                   
2021-08-19 23:05:10.968 T:6388    ERROR <general>:     
2021-08-19 23:05:10.968 T:6388    ERROR <general>: self.Views.remove_library(library_id)
2021-08-19 23:05:10.968 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.968 T:6388    ERROR <general>:   File "C:\Users\Brian\AppData\Roaming\Kodi\addons\plugin.video.emby-next-gen\emby\views.py", line 301, in remove_library
                                                   
2021-08-19 23:05:10.972 T:6388    ERROR <general>:     
2021-08-19 23:05:10.972 T:6388    ERROR <general>: whitelist = self.EmbyServer.library.sync.SyncData['Whitelist']
2021-08-19 23:05:10.973 T:6388    ERROR <general>:
                                                   
2021-08-19 23:05:10.973 T:6388    ERROR <general>: AttributeError
2021-08-19 23:05:10.973 T:6388    ERROR <general>: :
2021-08-19 23:05:10.973 T:6388    ERROR <general>: 'NoneType' object has no attribute 'sync'

 

 

...that's what I can see after log review. As mentioned I'll perform several tests but the main question still stands... Why new TV-Shows are not synced in the first place.

Edited August 20, 2021 by quickmic

[BaukeZwart 94]

28 minutes ago, quickmic said:

fyi, Kodi syncs content completely different while startup compared with runtime updates.

Runtime syncs uses the websocket connection (that's why you should test it). If websocket is not working, no updates will be pushed from Emby server BUT. It should sync on Kodi start. The plugin actively queries Emby sync data on Kodi startup and not waiting for data pushed from Emby server.

Could this also be the root cause of the sync issues I reported?
Can it be that when changing metadata with a 3th program like tmm in my case Emby is picking up the NFO changes but isn't sending a webhook so Kodi isn't updated until I do a Kodi restart.

EDIT: things like new added episodes are synced while tmm added changes aren't.

Edited August 20, 2021 by BaukeZwart

[quickmic 1477]

1 minute ago, BaukeZwart said:

Could this also be the root cause of the sync issues I reported?
Can it be that when changing metadata with a 3th program like tmm in my case Emby is picking up the NFO changes but isn't sending a webhook so Kodi isn't updated until I do a Kodi restart.

Not sure, currently I'm following the scenario from the log. Updates queued but paused due to platback -> quit Kodi -> restart Kodi.

As I can see, for some reason the startup sync was not initiated after that scenario...

I'm trying to design a solid test case and finally fix the issue.

 

On Kodi start (after Emby server connection bla) "[ retrieve changes ]" should be somewhere in the log. If not, startup sync was not initiated. This is currently the case after I followed Brian's approach.

 

[quickmic 1477]

@brian1709

Please try https://kodi.emby.tv/Public testing/plugin.video.emby-next-gen-6.0.4-build 30 (ex5).zip

[brian1709 9]

9 hours ago, quickmic said:

Is the websocket connection working? You can test this by sending a message from Emby server to Kodi client.

When did you add a TVShows? While Kodi was running or Kodi stopped?

web socket does not seem to be running

Shows are always added while kodi is offline, because I remux and transcode everything to a standard format and then move it all in one go once a day

9 hours ago, quickmic said:

he issue in the log came from Exit before repair sync was completed, not critical but I'll fix it anyway.

Also I can see in the log you watched a video while you triggered the repair sync. Syncs will paused while videos are playing. I'll check the repair sync code, currently I assume:

The delete content job is not paused while video playback. Therefore all the content was deleted (so far correct), next job: Updates are started but due to playback is was immediately paused.

According to the log, you never stopped the playback until Kodi exit

yes, I got tired off waiting for the repair so started watching a show and then just stopped Kodi when I was done, even though repair was not.

 

Of course it stopped updating again but that is probably due to the web socket 

Why that is not started I have no idea. Of course my DB only updates on start up (when it works) in any case since I don't add media while Kodi is running, but Progress and play next etc. updates all the time while I am using Kodi and that must also be using web socket, No?

 I am going to run the zip, restart emby and send you the log. How do we address the web socket issue?

kodi.log

Edited August 20, 2021 by brian1709

[brian1709 9]

Well I may be wrong. According to the log the web socket is started and I can see the 3 or 4 attempts of sending a message but never received it on the client side. Also the log says that "retrieve changes ' but the DB is not updated. I added a test media file and it just does not sync

Edited August 20, 2021 by brian1709

[brian1709 9]

Restarted the whole computer and now it is going fine and syncing perfectly. Maybe build 30 ex 5 did fix something

[brian1709 9]

Hi, just an update. Build 30ex5 has been working great so I decided to update it on my other machines as well.

Interesting issue has shown up there. I uninstall Emby for Kodi next gen, Clean out the Database and do an install on 30ex5. Everything goes fine, Movies sync, TV shows start and get about 15% through before it pops up with a " do you want to resume the previous update or sync..."

Where does it get that information from? It is a clean install with no previous DB. It then stops the sync if I say yes or goes a little more before it stops if I say no.

kodi.old.log

[quickmic 1477]

1 minute ago, brian1709 said:

Hi, just an update. Build 30ex5 has been working great so I decided to update it on my other machines as well.

Interesting issue has shown up there. I uninstall Emby for Kodi next gen, Clean out the Database and do an install on 30ex5. Everything goes fine, Movies sync, TV shows start and get about 15% through before it pops up with a " do you want to resume the previous update or sync..."

Where does it get that information from? It is a clean install with no previous DB. It then stops the sync if I say yes or goes a little more before it stops if I say no.

kodi.old.log 2.26 MB · 0 downloads

Please update to latest versions if you go with experimental. 30ex5 is already "damn old", it's a work in progress. I'm sure all those issues are already fixed.

Latest versions are always available here (page one), and a new version will be released soon :