Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 9 minutes ago, CBers said: It's been fascinating reading this thread as it has evolved. Congrats to you guys for putting this together Hey Keith, Hows it going? And Thanks, i really didn't do much, it's all chef!!
chef 3810 Posted September 10, 2021 Posted September 10, 2021 14 minutes ago, Cheesegeezer said: Wow Chef, you really wanted to be heard 3 of the same posts lol My Google pixel has turned out to be a paper weight after only three years 1
CBers 7450 Posted September 10, 2021 Posted September 10, 2021 15 minutes ago, Cheesegeezer said: Hey Keith, Hows it going? And Thanks, i really didn't do much, it's all chef!! Good thanks. You? Don't forget us on Discord 1 1
chef 3810 Posted September 10, 2021 Posted September 10, 2021 (edited) 8 hours ago, crusher11 said: Not that I can hear. One of the things we as a community need to do, is audit the data. It was something that I've always said needs to happen. Just get a rough idea of how many episodes were right in a season, and how many needed the users attention. For the season you mentioned it sounds like a couple of the episodes were wrong, most of the others that had acceptable data. Overall it worked. Is that a fair statement? Edited September 10, 2021 by chef
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 3 minutes ago, chef said: One of the things we as a community need to do, is audit the data. It was something that I've always said needs to happen. Just get a rough idea of how many episodes were right in a season, and how many needed the users attention. For the season you mentioned it sounds like a couple of the episodes were wrong, out of the others that had acceptable data. Overall it worked. Is that a fair statement? I have a feeling it's down to hardware, my test laptop is i5, 8GB Ram and intel gfx. Not very powerful has fallen short on a couple of episodes from 1 season and didn't even recognise that tricky episode i was testing. I will run this on my main server today... because I'm loco and have big cahunas lol. If it all goes pete tong... i'll just revert back to non beta and remove the plugin. But i will have the logs for us to figure out whats going wrong. @rbjtech can you update our roadmap please. Cheers 1
chef 3810 Posted September 10, 2021 Posted September 10, 2021 26 minutes ago, CBers said: Good thanks. You? Don't forget us on Discord chef#8353
neik 873 Posted September 10, 2021 Posted September 10, 2021 (edited) 1 hour ago, Cheesegeezer said: @chef @rbjtech @ebr @Luke Anyone know how to trigger another Scheduled Task in code. I can't seem to find the task in Emby for running the ThumbnailImageExtract Task. It used to be here... Emby/Emby.Server.Implementations/ScheduledTasks/ChapterImagesTask.cs / Chapters is pretty much complete. Chef I have switched the GetResult(string id) back now in my code and can pass the Episode Id to the ChapterEdit Method, so when running the Chapter Insertion Task, it will ONLY select your sequenced episodes to process. You will have to then run the thumbnailImageExtract task after running the chapter Insertion task I have just done a full test on my small Test library and everything is working as expected. Attached is 2.0.2.9 This build has been set as a pull request in Git, I'll let you check it and merge the request. You may also notice a new design for the Plugin Icon -- Sorry chef the old one just looked like it was from an 80's video game lol IntroSkip 2.0.2.9.zip 206 kB · 2 downloads Will this version add the intros into the chapter view or is it still WIP? Once that is added I would give this a go on my prod system and see how it goes. Edited September 10, 2021 by neik
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 1 minute ago, neik said: Will this version add the intros into the chapter view or is it still WIP? Once that is added I would give this a go on my prod system and see how it goes. yes. you have to run in this order Fingerprint task Detection task chapter insert task thumbnail image task
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) 32 minutes ago, Cheesegeezer said: @rbjtech can you update our roadmap please. Cheers Yep - np - let me catch up on what has been updated and I'll update and re-post. 2.0.2.9 is the latest right ? 7 minutes ago, neik said: Will this version add the intros into the chapter view or is it still WIP? Once that is added I would give this a go on my prod system and see how it goes. Yes it will add Intro data into the chapter view - but personally, I would still test first on an isolated Beta. It is, afterall, still in Beta .. Edited September 10, 2021 by rbjtech
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 @rbjtech do you know which version of ffmpeg i need to install to get the detection to work? I read it somewhere and the new beta has this version.
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) 6 minutes ago, Cheesegeezer said: @rbjtech do you know which version of ffmpeg i need to install to get the detection to work? I read it somewhere and the new beta has this version. Yes, latest beta has the correct Windows ffmpeg. I'm not 100% sure of the situation on other platforms - let me look up the Emby ffmpeg version - 2 secs. Windows Beta (.10) is using - ffmpeg version 4.4.0-emby_2021_08_23-g8f222573e9+4998 Edited September 10, 2021 by rbjtech
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 2 minutes ago, rbjtech said: Yes, latest beta has the correct Windows ffmpeg. I'm not 100% sure of the situation on other platforms - let me look up the Emby ffmpeg version - 2 secs. got it, don't worry the wife is watching Netflix on the main server PC so I'm nipping in under the radar from the laptop. I just copied the FFmpeg from laptop to the main server folder and run the fingerprint... looks like it's working now. 1
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 I have a proposal.... would it not be a good idea to.. Process the fingerprint, detection and chapters for each season and then move on to process the next. That way if the user stops it, then at least some of the episodes are populated. I would imagine that this would tie in nicely when you are asleep from 01:00 to 07:00 I have also been having a think, that we can now move onto generating the PlaybackProgress Flags in the plugin and set public accessors for the VideoOSD to use.
chef 3810 Posted September 10, 2021 Posted September 10, 2021 (edited) @Cheesegeezer look at the ItemAdded event and you'll see how to kick of tasks using ITaskManager. You just have to get the task by 'Name' using LINQ (FirstOrDefault), then call 'Execute'. You may need to create a new Cancellation token. Edited September 10, 2021 by chef
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) 28 minutes ago, Cheesegeezer said: I have a proposal.... would it not be a good idea to.. Process the fingerprint, detection and chapters for each season and then move on to process the next. That way if the user stops it, then at least some of the episodes are populated. I would imagine that this would tie in nicely when you are asleep from 01:00 to 07:00 I have also been having a think, that we can now move onto generating the PlaybackProgress Flags in the plugin and set public accessors for the VideoOSD to use. This came up wayyy back on Page 30 () as a possibly implementation .. It cropped as as a consequence of the FP dB becoming huge (Multi Gb in size) resulting in long dB lookup times. Chef then made changes to remove the FP data once the Detect for the Season had been successfully completed - ie you don't need the FP data anymore. I then suggested, as you have, why not round robin the tasks to do an FP for a show, then a Detect for a show (and now maybe a Chapter for a show) and then process the next show. This will not only will keep the FP Database size in check, but as you say, people will 'see' full results on shows that have been processed without completing the entire task. You can do this manually - simply by limiting the task to X minutes (ie interval) and move onto the next scheduled task, but it's messy. The good news is IntroSkip doesn't forget what it did - and will gracefully carry on post a resume - but it does seem a very unorthodox way of doing it vs doing it a show at a time. Anyway - let me test the latest - haven't had a chance to even load it up yet ! Edited September 10, 2021 by rbjtech
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 7 minutes ago, rbjtech said: This came up wayyy back on Page 30 () as a possibly implementation .. It cropped as as a consequence of the FP dB becoming huge (Multi Gb in size) resulting in long dB lookup times. Chef then made changes to remove the FP data once the Detect for the Season had been successfully completed - ie you don't need the FP data anymore. I then suggested, as you have, why not round robin the tasks to do an FP for a show, then a Detect for a show (and now maybe a Chapter for a show) and then process the next show. This will not only will keep the FP Database size in check, but as you say, people will 'see' full results on shows that have been processed without completing the entire task. You can do this manually - simply by limiting the task to X minutes (ie interval) and move onto the next scheduled task, but it's messy. The good news is IntroSkip doesn't forget what it did - and will gracefully carry on post a resume - but it does seem a very unorthodox way of doing it vs doing it a show at a time. Anyway - let me test the latest - haven't had a chance to even load it up yet ! If your fingerprinting and detection is already done, you just need to run the chapters and the thumbnail extraction tasks 1
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) 10 minutes ago, Cheesegeezer said: If your fingerprinting and detection is already done, you just need to run the chapters and the thumbnail extraction tasks Agreed - and on a small library - this is fine - but on a sizable library getting the FP 'complete' is a huge task. The dB will not have any FP's marked for deletion at this point - thus it remains potentially huge. The dB vacuum will only happen when the FP task is run again (as it's done at the start of the task). Once the Detect task starts, it marks the FP data for deletion as it goes - but of course, there is little point in this, as the FP task is complete anyway .. Run on a large library, and you'll see what I mean .. Edited September 10, 2021 by rbjtech
Cheesegeezer 3104 Posted September 10, 2021 Posted September 10, 2021 9 minutes ago, rbjtech said: Run on a large library, and you'll see what I mean .. Just running the FP now on my Main Server and only have 77 TV Shows, dunno how many episodes, but after 1hr of running the task i'm 29% in. I would say I have a small library lol
chef 3810 Posted September 10, 2021 Posted September 10, 2021 (edited) Maybe @Cheesegeezer has an idea about getting series to process fully before moving on, but because the task is split into two, it was fairly involved to process in that way. I only saw one way, and that was to combine the tasks into one behemoth. But, I would suggest that that is a bad idea. For users who are worried about the their library size, it would be easier to stop the tasks half way and run the second task. But!! Be aware that there is code in the second task that marks items as 'processed', which currently causes issues with stoping the fingerprint task before it is complete and running title sequence detection. That would have to be addressed before telling users that was a possibility. You can see where we create an 'unmatched' hashset to find out what hasn't been processed, and take from the actual library query the items we want. Edited September 10, 2021 by chef
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) - Big smile to see the chapters in a usable form ! Awesome work guys. A couple of little issues to report - mainly on Chapter 2+ items where I can potentially 'miss' content. Let me do some more testing and I'll report back. edit - scrap that, I tested on some episodes where I was manually adding chapters myself haha .. so far - looking 100%. I guess the only undesirable thing is having the 'Intro End' if there if a 'real' chapter 2 marking the same position already there. And having an 'Intro Start' at 0:00 as well as a 'Chapter 1' @ 0:00 As we don't want to mess about with the existing chapters - I guess there is not a lot we can do about that for the time being. Personally, I have no issues just deleting the duplicate chapter and renumbering the rest. We need to keep the IntroStart & IntroEnd for identification. Is this something we could add as an OPTION maybe ? Testing continues .. Edited September 10, 2021 by rbjtech
Sammy 790 Posted September 10, 2021 Posted September 10, 2021 This is taking 4-eh-ver.. You'll probably drop a few new versions before it is done.
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) Latest updates below - Amazing work from @chef and @Cheesegeezer to get #5 out the door so quickly. (added Required Version Number - as development is happening fast, many testers are on older versions..) # Item Description Stage Latest Update Version Required Needs Core Updates ? ETA 1 FingerPrint(Chroma) Creates a template based on a percentage of the Audio runtime. Ready More efficient use of dB but good enough for release when used with scheduler 2.0.2.8 No Complete 2 Detection Detects the 'repeating Audio' vs other episodes and calculates the IntroStart and IntroEnd Ready Possibly more accuracy but certainly good enough at the moment with 90-95% 2.0.2.8 No Complete 3 Schedule Due to the Detection needing the FP database, a schedule needs to be run to ensure the data is available. Ready but needs work Working now but needs manual config 2.0.2.8 No ? 4 Scaling Depending on library size, these tasks can take days, even weeks - so it's important it's broken into manageable chunks. Ready but needs work Needs more testing on low powered NAS devices 2.0.2.8 No ? 5 Manual Skip The IntroStart/IntroEnd can be added into the existing chapter points without needing Core development work Testing 2.0.2.9 No Complete 6 IntroSkip - Manual Detection of an 'IntroStart/IntroEnd (in emby Chapters?) should bring a 'Skip Intro?' button to the front of the playback screen. Design Yes ? 7 IntroSkip - Auto Assuming accuracy is good enough, then #6 could also be automatic if given the option. Concept Yes ? 8 Community dB A community dB of existing IntroStart/End points Concept No ? 9 Specials Handling of Special Episodes - both In Season and Out of Season Concept No ? 10 Meta-Data Refresh What happens when Metadata is refreshed - do we keep chapter points ? Design No ? 11 External chapter file Write all chapters (incl new Intro) to an external XML or Text File Concept No ? ? Edited September 10, 2021 by rbjtech 1
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) Testing chapters completed on 7 shows - using both real and auto chapters. Results are it looks 100% solid and it is inserting chapters into the correct sequence without issues - sometimes not appearing until after Chapter 3 or 4 in one or two cases. I'd personally like to see the unnecessary/duplicate chapters removed (in the Emby dB, the original file is never touched) - but that's just me - thoughts welcome. Edited September 10, 2021 by rbjtech
chef 3810 Posted September 10, 2021 Posted September 10, 2021 You guys know that there is API endpoints for all the sequence data right? If you open the swagger, you can test it out. Just wanted to make sure that was clear.
rbjtech 5284 Posted September 10, 2021 Posted September 10, 2021 (edited) One to add - Could you change the log Class for the Chapter Insertion to use 'Intro Skip' instead of App - then we keep all the logging together Edited September 10, 2021 by rbjtech sorry it's 'Intro Skip' not 'IntroSkip' ..
Recommended Posts