Jump to content

Kodi MediaImport Project


LongMan

Recommended Posts

Montellese
8 hours ago, strech said:

@LongMan,

would it also be possible to install your plugin also by adding the repo manuelly? So i could install it on v19re1? Or why a special branch is needed?

Because media importing requires a lot of additional code in Kodi's core. Without it the media import add-ons aren't even recognized by Kodi and the interface they use to provide imported items doesn't exist.

I'll try to rebase it onto RC1 ASAP.

Link to comment
Share on other sites

  • 2 weeks later...
LongMan

Updated Builds based on Kodi v19 RC1 posted here

Edit: Please remember to delete your previous MyVideos120.db because there has been changes to the database layout. Also remember to install the latest versions of the Emby / Plex media importers because there has been a breaking change in the add-on interface:

  • Emby Media Importer: 0.0.10
Edited by LongMan
  • Like 2
  • Thanks 1
Link to comment
Share on other sites

LongMan

New builds are posted on https://github.com/Montellese/xbmc/wiki/Downloads  (still based on Kodi v19 RC1). A bug was introduced during the rework. The change was reverted and a better solution found without regressions.

mediaimporter.emby has been updated 0.0.11.

This contains

  • a fix for authenticating using Emby Connect
  • the possibility to authenticate with an empty password
  • a new advanced option to generate a new device ID

NB. One can now run the initial sync on a more powerful device and copy the database. Then use the new Generate New DeviceID to get it it's own identity on the server. This means that it may only be necessary to run the initial sync once if you are using multiple devices.

Also, could you tell us if you are using these builds as daily drivers and provide some feedback.

 

Cheers,

LongMan

Edited by LongMan
Link to comment
Share on other sites

LongMan

From Montellese,

Quote

A question to Emby users: Would it also be useful for Emby servers to be able to change the URL of a discovered / added media provider?

 

Link to comment
Share on other sites

  • 3 weeks later...
TeamB
42 minutes ago, LongMan said:

New builds

  • Rebased onto Kodi Matrix (Final)
  • Greatly improved the performance of adding media items retrieved from a media provider to the database (only tested with SQLite!)

Any idea how close this is to being released with the base kodi?

Also where is the source checked in?

Link to comment
Share on other sites

LongMan

The release is based on

https://github.com/Montellese/xbmc/tree/media_import_19.0-Matrix_performance

further performance improvements are in another branch

https://github.com/Montellese/xbmc/tree/media_import_19.0-Matrix_performance_async

He is looking at optimizations now, and he is cherry picking improvements that are not MediaImport specific.

It is probably best if @Montellese gives a timeline as his time is the limiting factor. However, having a couple of video addons to test would be nice. At present there is only one and it is country specific

 

Edit: mediaimporter.emby

Edited by LongMan
Add link to importer source code
Link to comment
Share on other sites

LongMan

@TeamB,

Could you please chime in here. Montellese is looking at performance of the MediaImport addons looking for input into how to modify the python interface. I know you were involved with Emby for Kodi previously and has a lot of experience from EmbyCon. Please if you can, chime in.

@quickmic. If possible could you comment as well please. This could affect Emby Next-Gen

Link to comment
Share on other sites

LongMan

New test builds which contain

  • small performance improvements to the Python API which should benefit the performance of retrieving media items from servers
  • major performance improvements (factor 4) when adding actors to the video database


Downloads: https://github.com/Montellese/xbmc/wiki/...#downloads
Changelog: https://github.com/Montellese/xbmc/wiki/...ormance-v2

Link to comment
Share on other sites

LongMan

@TeamB,

Could you test Embycon on this build to see if you get the performance boost that is referenced.

Link to comment
Share on other sites

Montellese
3 hours ago, TeamB said:

yeah I can have a look, do you happen to know which of the functions were optimized?

In general any Python API method which takes a Python list / C++ std::vector or a Python dict / C++ std::map / Dictionary. The bigger the list / dict the more performance gain. In my performance measurements it was most prominent on ListItem.setCast().

Link to comment
Share on other sites

TeamB

@LongMan @Montellese

I had some time today to run some performance comparisons between the official build and the special build above.

In general it is faster, especially for setCast as you mentioned but only marginally faster for some of the other function on ListItem that I use.

The following data from 3 runs (to get an average) to extract python profile data from both the official build and the special build, the most important are the % numbers.
If you want the raw profile data I still have it and can send it to you if you want.

% Performance improvements:
setProperties : 2.9%
setInfo : 7.9%
setCast : 23%
setArt : 1.7%

Average time in seconds for ListItem functions

Official build Kodi 19
setProperties : 1.554
setInfo : 0.405
setCast : 0.291
setArt : 0.175

Special build Kodi 19
setProperties : 1.509
setInfo : 0.373
setCast : 0.224
setArt : 0.172

Data from runs

Official Kodi 19 : Run 01

    71400    1.502    0.000    1.502    0.000 {method 'setProperties' of 'xbmcgui.ListItem' objects}
    10200    0.403    0.000    0.403    0.000 {method 'setInfo' of 'xbmcgui.ListItem' objects}
    31700    0.338    0.000    0.338    0.000 {method 'addStreamInfo' of 'xbmcgui.ListItem' objects}
    10200    0.284    0.000    0.284    0.000 {method 'setCast' of 'xbmcgui.ListItem' objects}
    10200    0.170    0.000    0.170    0.000 {method 'setArt' of 'xbmcgui.ListItem' objects}
    10200    0.011    0.000    0.011    0.000 {method 'setRating' of 'xbmcgui.ListItem' objects}
    10200    0.005    0.000    0.005    0.000 {method 'setContentLookup' of 'xbmcgui.ListItem' objects}
    
Official Kodi 19 : Run 02
    71400    1.596    0.000    1.596    0.000 {method 'setProperties' of 'xbmcgui.ListItem' objects}
    10200    0.414    0.000    0.414    0.000 {method 'setInfo' of 'xbmcgui.ListItem' objects}
    31700    0.350    0.000    0.350    0.000 {method 'addStreamInfo' of 'xbmcgui.ListItem' objects}
    10200    0.299    0.000    0.299    0.000 {method 'setCast' of 'xbmcgui.ListItem' objects}
    10200    0.180    0.000    0.180    0.000 {method 'setArt' of 'xbmcgui.ListItem' objects}
    10200    0.012    0.000    0.012    0.000 {method 'setRating' of 'xbmcgui.ListItem' objects}
    10200    0.006    0.000    0.006    0.000 {method 'setContentLookup' of 'xbmcgui.ListItem' objects}

Official Kodi 19 : Run 03
    71400    1.565    0.000    1.565    0.000 {method 'setProperties' of 'xbmcgui.ListItem' objects}
    10200    0.397    0.000    0.397    0.000 {method 'setInfo' of 'xbmcgui.ListItem' objects}
    31700    0.356    0.000    0.356    0.000 {method 'addStreamInfo' of 'xbmcgui.ListItem' objects}
    10200    0.289    0.000    0.289    0.000 {method 'setCast' of 'xbmcgui.ListItem' objects}
    10200    0.176    0.000    0.176    0.000 {method 'setArt' of 'xbmcgui.ListItem' objects}
    10200    0.011    0.000    0.011    0.000 {method 'setRating' of 'xbmcgui.ListItem' objects}
    10200    0.005    0.000    0.005    0.000 {method 'setContentLookup' of 'xbmcgui.ListItem' objects}

Special Build Kodi 19 : Run 01
    71400    1.516    0.000    1.516    0.000 {method 'setProperties' of 'xbmcgui.ListItem' objects}
    10200    0.379    0.000    0.379    0.000 {method 'setInfo' of 'xbmcgui.ListItem' objects}
    31700    0.333    0.000    0.333    0.000 {method 'addStreamInfo' of 'xbmcgui.ListItem' objects}
    10200    0.228    0.000    0.228    0.000 {method 'setCast' of 'xbmcgui.ListItem' objects}
    10200    0.174    0.000    0.174    0.000 {method 'setArt' of 'xbmcgui.ListItem' objects}
    10200    0.012    0.000    0.012    0.000 {method 'setRating' of 'xbmcgui.ListItem' objects}
    10200    0.005    0.000    0.005    0.000 {method 'setContentLookup' of 'xbmcgui.ListItem' objects}
    
Special Build Kodi 19 : Run 02
    71400    1.512    0.000    1.512    0.000 {method 'setProperties' of 'xbmcgui.ListItem' objects}
    10200    0.369    0.000    0.369    0.000 {method 'setInfo' of 'xbmcgui.ListItem' objects}
    31700    0.326    0.000    0.326    0.000 {method 'addStreamInfo' of 'xbmcgui.ListItem' objects}
    10200    0.220    0.000    0.220    0.000 {method 'setCast' of 'xbmcgui.ListItem' objects}
    10200    0.170    0.000    0.170    0.000 {method 'setArt' of 'xbmcgui.ListItem' objects}
    10200    0.012    0.000    0.012    0.000 {method 'setRating' of 'xbmcgui.ListItem' objects}
    10200    0.005    0.000    0.005    0.000 {method 'setContentLookup' of 'xbmcgui.ListItem' objects}
    
Special Build Kodi 19 : Run 03
    71400    1.498    0.000    1.498    0.000 {method 'setProperties' of 'xbmcgui.ListItem' objects}
    10200    0.372    0.000    0.372    0.000 {method 'setInfo' of 'xbmcgui.ListItem' objects}
    31700    0.335    0.000    0.335    0.000 {method 'addStreamInfo' of 'xbmcgui.ListItem' objects}
    10200    0.225    0.000    0.225    0.000 {method 'setCast' of 'xbmcgui.ListItem' objects}
    10200    0.171    0.000    0.171    0.000 {method 'setArt' of 'xbmcgui.ListItem' objects}
    10200    0.011    0.000    0.011    0.000 {method 'setRating' of 'xbmcgui.ListItem' objects}
    10200    0.005    0.000    0.005    0.000 {method 'setContentLookup' of 'xbmcgui.ListItem' objects}

 

 

Edited by TeamB
  • Like 1
Link to comment
Share on other sites

TeamB

Forgot to mention, this testing was done on a fast i7 windows 10 machine.

Any chance of getting that 23% performance boost on the other functions?

Link to comment
Share on other sites

Montellese

Thanks for taking the time to perform these measurements!

6 minutes ago, TeamB said:

Forgot to mention, this testing was done on a fast i7 windows 10 machine.

Any chance of getting that 23% performance boost on the other functions?

Not with the same approach. ListItem.setCast() was easy to improve because it usually takes a large data set so pre-allocating the necessary data and better handling the memory needed provides a major improvement. For other methods which only take small amounts of data there is no gain in doing this.

Link to comment
Share on other sites

TeamB

Ok I understand.

I wonder if there is any benefit in passing in ALL the ListItem data as a json string to a constructor in one go, when you are building a listitem for display or storage you know everything about it so you have it all, no need to call this function then that one, just pass it all at once and convert the json to build the list item in c++

Link to comment
Share on other sites

quickmic

Just curious, cause I did some research in the past about that matter but not spent too much time...

Do you know is it possible to combine all those API call into one single command?

setProperties, setInfo, setCast, setArt

I mean, filling a dict with ALL info and perform a single API setProperties call?

 

Edited by quickmic
Link to comment
Share on other sites

alscaldas

Hello there.

I didn't know this project. In fact I found it by chance when looking for Emby for Kodi alternatives, but I took a look at the wiki I liked very much the way it works, so I decided to try it.

I own a RockPro64 and tried to install LibreELEC Matrix, but I there's no MediaImport repository neither enabled nor disabled to choose. The same goes with Kodi Matrix installed on a Manjaro setup. Is it something I'm doing wrong? I didn't found a external repository to add and activate, which makes me suppose it's something that must be compiled with Kodi, right? Are there any plans to make the support to the RockPro64 (rk3399 based) available?

Thanks.

Link to comment
Share on other sites

LongMan

@alscaldas. No you have not done anything wrong. These are separate builds from vanilla Kodi. The builds are found here. Currently we don't have a place to host LibreELEC builds so there are none linked there. However, I build LibreELEC for myself and if you want I could run a build for Rockchip RK3399 for you. Let me know.

 

Cheers,

LongMan

Link to comment
Share on other sites

alscaldas
1 minute ago, LongMan said:

@alscaldas. No you have not done anything wrong. These are separate builds from vanilla Kodi. The builds are found here. Currently we don't have a place to host LibreELEC builds so there are none linked there. However, I build LibreELEC for myself and if you want I could run a build for Rockchip RK3399 for you. Let me know.

 

Cheers,

LongMan

I'd love to. In fact it wouldn't have to be necessarily LibreELEC. It could be Kodi, if you already compile it the way you do with LibreELEC (the way is less troublesome to you). Would it support hardware acceleration as the official builds do?

 

Thanks for you interest.

Link to comment
Share on other sites

LongMan

I have started the Rockchip RK3399 build. It is the first time running that project so the depencies are being downloaded. It make an hour+ because some of the hosting servers are slow normally build Generic and Ras Pi 4, and I have builds for those. When the build is complete I will post it.

Did you see the Kodi (MediaImport) builds in the link? There are builds for x86 / x64, ARMv7a / ARM64v8a and OSX /  tvOS.

  • Like 1
Link to comment
Share on other sites

alscaldas
1 minute ago, LongMan said:

I have started the Rockchip RK3399 build. It is the first time running that project so the depencies are being downloaded. It make an hour+ because some of the hosting servers are slow normally build Generic and Ras Pi 4, and I have builds for those. When the build is complete I will post it.

Did you see the Kodi (MediaImport) builds in the link? There are builds for x86 / x64, ARMv7a / ARM64v8a and OSX /  tvOS.

Thank you. It's very kind of you.

Yep, I'd already seen the links you're referring to but there's no version available to RockPro64 (the closest would be maybe the ARRMv7a or ARM64v8a, but they're for Android).

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...