Jump to content


Photo

Library update issues


  • Please log in to reply
14 replies to this topic

#1 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 28 November 2019 - 08:17 AM

Hello,
 
The following issues began around 6 days ago but due to thetvdb I decided to wait...
 
1) Emby server responds with 404 not found, has endpoint `Library/Series/Updated` been removed ?
Info HttpServer: HTTP Response 404 to ::1. Time: 1ms. http://localhost:8096/emby/Library/Series/Updated
 
Thanks.

Edited by JackDandy, 28 November 2019 - 11:22 AM.


#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 28 November 2019 - 11:04 AM

It hasn't been removed. What http method did you use?

#3 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 28 November 2019 - 11:21 AM

Yup, something seems to have changed...
 
Seems GET is no longer allowed on that endpoint (due to 404 in response).
 
I changed request type to POST and URL to `Library/Media/Updated` where `System/Info/Public/Version` is greater than 4.3.0.0 to fix the issue, seems okay in the logs, and the response is as expected.
 
Swagger reports that `Series` is deprecated in favour of `Media`.

Would be nicer if it says what version it was removed from, or was going to be removed from. So client apps can react according to EMBY version they are connected to.

Also, just some indication in the changelog or "whats new" announcement so that client devs can keep up with you would be helpful.

Thanks again.


Edited by JackDandy, 28 November 2019 - 01:20 PM.


#4 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 28 November 2019 - 12:54 PM

Sorry I must have forgotten about this one. Nonetheless the library media updated one is there better route anyway because it's based on the file path.

#5 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 28 November 2019 - 01:10 PM

Nonetheless the library media updated one is there better route anyway because it's based on the file path.

Oh i didn't notice that the params had changed too, figures now why the endpoint is deprecated.

However, dict('TvdbId': 'tvdb_id') was the old parameter, but this has changed to a full path.

{
"Path": "string",
"UpdateType": "string"
}

1) What can be the values for key 'UpdateType'?

2) Can the client no longer specify an EMBY series id and have it use the paths and mapped paths that it already knows about?

3) Does this mean the client has to maintain a set of mapped paths?

4) in the old 'Series' endpoint, the client could send no params and Emby would simply update the libraries, how to do this now?

 

I feel nervous now, i'm gonna revert from this 'Media' endpoint to the still functioning 'Series' endpoint and use that (at least a http POST fixes the immediate terminal failures) until these quite impacting changes are developed further. It seems that this endpoint change currently adds quite a bit of work for user and developer.


Edited by JackDandy, 28 November 2019 - 01:32 PM.


#6 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 28 November 2019 - 01:30 PM

Series id isn't a great way of doing it. What if the server doesn't have that id in the database, or what if you have multiple series folders with the same id. That's why a more primitive approach is better. Just tell the server what files have changed.

Try omitting updatetype.

#7 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 28 November 2019 - 01:47 PM

"Try omitting updatetype." 

 

ok, so that arg is optional but what is it for and what values can it take, or is this documented somewhere for me to read?


Edited by JackDandy, 28 November 2019 - 01:47 PM.


#8 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 28 November 2019 - 01:50 PM

I don't have that in front of me right now but I think swagger should list the possible values. It's so you can tell the server if the file was added, modified or deleted.

#9 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 28 November 2019 - 06:59 PM

swagger should list the possible values.


Unfortunately, Swagger has not documented possible input values for this, it simply shows the arg names.

#10 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 28 November 2019 - 07:32 PM

Ok, I'll take a look at that. But for now you can omit it. Thanks.



#11 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 29 November 2019 - 06:38 AM

1) What if the server doesn't have that id in the database,
2) or what if you have multiple series folders with the same id.
3) That's why a more primitive approach is better. Just tell the server what files have changed.

To your q:1 why wouldn't it have an id, it always has. Of course, that id is TVDBid, and it is perhaps this id that you are trying to move away from, given recent events. But those recent bumps in the road are now settling and their API is returning more and more data on v3. To the point where we have been able to fully remove v1 support now that finally, v3 carries all of the thumb data. Also, TVDB is still by far the most comprehensive and uptodate source to the majority of shows - in our test results.  

  

To your q:2 not sure why this would occur (hasn't ever here) so maybe this is a hypothetical rather than practical concern, but yeah, perhaps scan all related paths and mapped paths for this use case.  

 

To your 3: I'm not so sure this "more primitive approach" is better than current Emby because it requires Emby to string match a supplied path to a show. A string comparison is more prone to errors and raises other issues (e.g. Lin/Win path differences, Win long paths, mapped paths, network/remote/UNC paths, syncing changes to show path between client and Emby); it's also more processor taxing and way less efficient than the far more direct ID approach.

 

If the '/Media/Updated' with param 'Path' is pushed and the other endpoint removed, we will fall back to requesting Emby scan the entire library every time a new media file is available, that way, less for us to worry over - of course this means a substantial backstep in terms of performance as SSDs/HDDs will be far more busy - but a reliable trusted outcome is the key priority over struggling with inevitable path matching issues.

 

Series id isn't a great way of doing it.

Up to now, `/Library/Series/Updated` using TvdbId POST param has proven flawless for our users, and we can't rock the boat with this big of a change leading up to xmas. The other small issue we found is now handled by using POST for a full library scan instead of GET. I guess, as the docs are not yet clear and binding that things are still subject to change, so we wait for your direction  :)


Edited by JackDandy, 29 November 2019 - 11:00 AM.


#12 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 29 November 2019 - 11:23 AM

Those may not happen for you but as we do have to support a large audience these sort of things come up.

Plus the api based on path will work generally for any kind of media.

#13 JackDandy OFFLINE  

JackDandy

    Newbie

  • Members
  • 7 posts
  • Local time: 05:39 PM

Posted 29 November 2019 - 01:39 PM

Those may not happen for you but as we do have to support a large audience these sort of things come up.

Yup, I appreciate that, um, not just me, but our audience too. And most of us API accessing audience has used your existing methods with ID for years without issues - to break this now is odd.

  

Plus the api based on path will work generally for any kind of media.

That doesn't address my highlighted concerns. This seems to not have been thought through from client code perspective... for example, how is client code to know that the path for show named "example" has been found by Emby to be a valid path for the actual show "example" ?
 
Anyway, we will simply do as with Kodi, and just request Emby to refresh everything - such a shame, Emby was the most i/o efficient with the ID system.


Edited by JackDandy, 29 November 2019 - 01:56 PM.


#14 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 29 November 2019 - 02:11 PM

Ok now that i'm able to look at this, this is actually still there:

Library/Series/Updated

So if you really want to do it that way, just send a post. it was never removed.



#15 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 142516 posts
  • Local time: 12:39 PM

Posted 29 November 2019 - 02:29 PM

 

 

That doesn't address my highlighted concerns. This seems to not have been thought through from client code perspective... for example, how is client code to know that the path for show named "example" has been found by Emby to be a valid path for the actual show "example" ?

 

There's still some refinement needed there, it could be handled via settings on either side, client or server, but it's nothing that can't be figured out. Point being, if a file is added or updated, that's all the server needs to know and should know. Then it can just use the regular scanning processes. By adding a dependency on metadata, you're reducing the number of people that it will end up working for. Remove that dependency and make it more primitive, and then it will work for everyone.

 

For example, what if someone edited their series in emby and removed the tvdb id, or any other kind of user error that would result in the tvdb id being removed. Then they're going to complain that this isn't working, and the two of us will be the ones that get stuck with troubleshooting it. If you want to tell me this shouldn't happen, you're right, it shouldn't, but it does and it ends up eating up a lot of our time.

 

And then what about adding a new series, it's currently not possible with the way you're doing to notify the server. Look at the sequence of events: Create new series folder + episode files -> notify emby server of new series based on tvdb id. But the server doesn't know about the new series folder yet, therefore there's nothing to match and all we can do is run a full library scan. On the other hand if you use the api to notify about the new series folder, then all we have to do is scan that one folder.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users