Plugin: Export VODS from .m3u - split m3u livetv, tvshows, movies (Going BETA!!)

[mickle026 650]

42 minutes ago, jaspalgill said:

If I upgrade to .170 but keep the directory output from .161 and

then the strm folders get updated fine using the new logic.

If I then re-enable the M3U extraction they work but then the strms stop being updated. If I delete them then they are lost and won't be re-created.

not quite understanding what this means

44 minutes ago, jaspalgill said:

disable the M3U extraction

This is what changed at that time, so not re-writing files that exist.

Quote

If I delete them then they are lost and won't be re-created.

if you point to a new folder are they created again?

 

[goin3d 4]

Currently, have Japanese, Chinese, and Arabic selected with the latest version and it they are still written out for me:

EDITED:  my fault!  had Koren and Chinese confused.  

Is it possible to add കമപളങങി നൈററ‌സ - 2019 (I think it's Hindi?)

Edited May 2, 2025 by goin3d

[Apotropaic 41]

1 hour ago, mickle026 said:

I would check first if they are being filtered incorrectly.
Are they being identified as Live TV for example and you don't have output for that checked?

Any chance you can send me your m3u?  There is an option to write it obfuscated now.
 

not quite understanding what this means

  1 hour ago, jaspalgill said:

disable the M3U extraction

Just as I hit submit I came to the same conclusion! Went to the chippy so just left it and was about to reboot the server when the file writer process kicked in and the strms have been written this time, correctly identified as Movies and TV Shows 

I'm going to give it another go with everything turned on in an empty folder structure, maybe in my new server it's only ever allocating itself 1-2gb so it's just taking longer than expected, looking as if it's not doing anything.

When I meant disabling the M3U extraction, I just meant when it splits the playlists into separate Movies, TV Shows, Live TV M3U's.

 

[mickle026 650]

1 hour ago, goin3d said:

s it possible to add കമപളങങി നൈററ‌സ - 2019 (I think it's Hindi?)

Added Malayalam (India) with some others
Protector of deep desires - കമപളങങി നൈററ‌സ

 

 

 

 

Edited May 2, 2025 by mickle026

[BigIrn 10]

Anyone having issues where they can't turn GENRE off from outputting the streams even though it's not checked? @mickle026

[mickle026 650]

22 minutes ago, BigIrn said:

Anyone having issues where they can't turn GENRE off from outputting the streams even though it's not checked? @mickle026

You must have had use tmbd api turned on, and populating the genres, I have added a check to null it if the checkbox is not selected.

 

 

VODtoSTRMscheduledTask.dll

Edited May 2, 2025 by mickle026

[sundevil67 28]

19 hours ago, mickle026 said:

@sundevil67as suspected, they are all detected as Live TV urls

For testing you can enable Save Live TV output because yours are all tv shows anyway.  It will output them all to a Live TV folder, but they wont get mixed if you are never loading a live tv m3u file.  Its only a folder name! it wont make any difference to the content

I ran your  m3u here (the one you sent via pm) and they detect properly as tv shows with that turned on

you only need your first url in the first url box, it will download the rest automatically, you dont need to use url 2 or url3
https://****************/1 

A new build is attached - can you enable this?

It will output an obfuscated split m3u files

 

Internally they should look like this

#EXTM3U
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E01
https://*************/*************/*************/tvshow/tt8332798/1/1
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E02
https://*************/*************/*************/tvshow/tt8332798/1/2
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E03
https://*************/*************/*************/tvshow/tt8332798/1/3
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E04
https://*************/*************/*************/tvshow/tt8332798/1/4
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E05
https://*************/*************/*************/tvshow/tt8332798/1/5
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E06
https://*************/*************/*************/tvshow/tt8332798/1/6
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E07
https://*************/*************/*************/tvshow/tt8332798/1/7
#EXTINF:-1 tvg-id="tt8332798" tvg-name="tt8332798" tvg-type="tvshows" group-title="Long Bright River (2025)" ,Long Bright River (2025) S01 E08
https://*************/*************/*************/tvshow/tt8332798/1/8
#EXTINF:-1 tvg-id="tt36108218" tvg-name="tt36108218" tvg-type="tvshows" group-title="Soviet Storm: WWII in the East (2025)" ,Soviet Storm: WWII in the East (2025) S01 E01
https://*************/*************/*************/tvshow/tt36108218/1/1
#EXTINF:-1 tvg-id="tt36108218" tvg-name="tt36108218" tvg-type="tvshows" group-title="Soviet Storm: WWII in the East (2025)" ,Soviet Storm: WWII in the East (2025) S01 E02
https://*************/*************/*************/tvshow/tt36108218/1/2
#EXTINF:-1 tvg-id="tt36108218" tvg-name="tt36108218" tvg-type="tvshows" group-title="Soviet Storm: WWII in the East (2025)" ,Soviet Storm: WWII in the East (2025) S01 E03
https://*************/*************/*************/tvshow/tt36108218/1/3
#EXTINF:-1 tvg-id="tt36108218" tvg-name="tt36108218" tvg-type="tvshows" group-title="Soviet Storm: WWII in the East (2025)" ,Soviet Storm: WWII in the East (2025) S01 E04
https://*************/*************/*************/tvshow/tt36108218/1/4
#EXTINF:-1 tvg-id="tt36108218" tvg-name="tt36108218" tvg-type="tvshows" group-title="Soviet Storm: WWII in the East (2025)" ,Soviet Storm: WWII in the East (2025) S01 E05
https://*************/*************/*************/tvshow/tt36108218/1/5

This will allow you to send me a full version of all the parts in  one file.

 

VODtoSTRMscheduledTask.dll 911 kB · 3 downloads

 

 

I still can't get it to write any strm files

ScheduledTask-[M3UToStrm]_02.May.2025-17.13.03.txt

[goin3d 4]

This is really strange....with the latest build i am unable to change and save any config updates.  i click the save button and nothing happens, tried rebooting, etc. and same result.  Is anyone else experiencing this?

[mickle026 650]

One of the language selectors wasn't set in the html, I had missed it.  So the html/JS threw a Query Selector error when trying to save.

Here it is fixed

VODtoSTRMscheduledTask.dll

[mickle026 650]

10 hours ago, sundevil67 said:

I still can't get it to write any strm files

ScheduledTask-[M3UToStrm]_02.May.2025-17.13.03.txt 16.67 kB · 1 download

Exception in Task.Run: The request was cancelled due to the configured HttpClient.Timeout of 100 seconds elapsing.

Failed at file 18 downloading, file 18 took over 100 seconds wait time.

The plugin didn't cancel it at 100 seconds, I am assuming that the remote server probably did - there is no message from the system, only the one above.
Pretty sure somewhere you mentioned there were around 30 files to download?

Edited May 3, 2025 by mickle026

[goin3d 4]

Not saving still for me at least, on the latest version

[mickle026 650]

22 minutes ago, goin3d said:

Not saving still for me at least, on the latest version

IN Firefox,

F12 and open the browser console.

Click save on the plugin page and see if there is an error.  If it does, on the right will be a filename, and line numbers.

Like this: Configuration xxx:174 , the line number, click it and it will show what it failed on in the JavaScript.

If it says Vietnamese anywhere then it's likely your browser has cached the page.  Either clear the cache or hold shift while reloading the page.

If it's different let me know.

[sundevil67 28]

4 hours ago, mickle026 said:

Exception in Task.Run: The request was cancelled due to the configured HttpClient.Timeout of 100 seconds elapsing.

Failed at file 18 downloading, file 18 took over 100 seconds wait time.

The plugin didn't cancel it at 100 seconds, I am assuming that the remote server probably did - there is no message from the system, only the one above.
Pretty sure somewhere you mentioned there were around 30 files to download?

Yes; those URLs go up to /29 I believe.. If I remove the /1 altogether, it looks like it will increment & keep downloading files at the subsequent URLs, /2, /3, etc. Is that just always going to hold it up? It'd be great to do them all in one shot, but I'd be happy doing 1 or 3 at a time at this point just to get it working at all. Not too many shows are going to have a creation date old enough to make it an issue.. In other words, the only ones that I'd probably have to keep "in sync" are 5-6 & under.

[mickle026 650]

16 minutes ago, sundevil67 said:

Yes; those URLs go up to /29 I believe.. If I remove the /1 altogether, it looks like it will increment & keep downloading files at the subsequent URLs, /2, /3, etc. Is that just always going to hold it up? It'd be great to do them all in one shot, but I'd be happy doing 1 or 3 at a time at this point just to get it working at all. Not too many shows are going to have a creation date old enough to make it an issue.. In other words, the only ones that I'd probably have to keep "in sync" are 5-6 & under.

Currently the plugin will always download from file.1 regardless of what URL number you put.

I can alter it to dect the file number from the URL and download from there but perhaps I ought to do that and add a delay between each file downloaded like 5 seconds, maybe the concurrent download is the issue, a delay will likely help.

[mickle026 650]

v1.0.0.174

6 hours ago, sundevil67 said:

Yes; those URLs go up to /29 I believe.. If I remove the /1 altogether, it looks like it will increment & keep downloading files at the subsequent URLs, /2, /3, etc. Is that just always going to hold it up? It'd be great to do them all in one shot, but I'd be happy doing 1 or 3 at a time at this point just to get it working at all. Not too many shows are going to have a creation date old enough to make it an issue.. In other words, the only ones that I'd probably have to keep "in sync" are 5-6 & under.

ApolloTV - TV Shows

I have added a retry mechanism, up to 5 times per file
I have added a delay between download attempts - each attempt waits a little longer than the last.
I have changed the url code to download from the number you specify on the end of your url string, if no number is specified then it will start at no.1
I have added a failed concurrency check (failed files in a row)
Timeout is now set at 200 seconds per attempt

currently it will try to download the parts past the number you specify, but after 5 failed attempts it will move on to the next part, so if there are 29 parts it will try part 30 before stopping. Part 30 should be a 404 not found error, it will stop downloading at a 404 error and start the processing.  404 should be there are no more files?
If there a two files in a row that fail 5 attempts to download, it will give up downloading and start processing.  (fall back if there is not a 404 error - but will take 10 attempts and 2.35 mins + timeouts extra to achieve this failure before continuing) - worst case with timeouts is just short of 10 mins extra.

This might increase the download times, but should be a lot more robust.
 

 

VODtoSTRMscheduledTask.dll

Edited May 3, 2025 by mickle026

[BigIrn 10]

On 5/1/2025 at 4:56 PM, mickle026 said:

OK, I think all your requests are now complete

@goin3d

Now an option whether or not to re-write files that differ, there is not a huge lot I can do here.  I have created a test for files that exist, if they do it wont overwrite them, but what is left is these duplicate entries that have a different url.
You choose either to update urls or not.  This will affect if anything else is updated to a new url location.

So now if the output file already exists you can choose by unchecking the box, not to write the second file with a different url over it, this will affect any other urls that have changed.

01.May.2025 - 14:38:41 Debugging: [scanM3UFileAsync] m3uContents Contains: 26 Items.
01.May.2025 - 14:38:41 -----------------------[ Updating Output Folder ]-------------------------------
01.May.2025 - 14:38:41 Debugging: Setting Filesystem Update to 12 of 16 cores/threads to prevent system freeze.
01.May.2025 - 14:38:41 Debugging: Remove Obsolete Files: True.
01.May.2025 - 14:38:41 Debugging: Files to check: 15.
01.May.2025 - 14:38:41 Debugging: Remove Obsolete Files - Progress: 7.69%, Memory used: 26 MB.
01.May.2025 - 14:38:41 Debugging: Remove Obsolete Files - Progress: 46.15%, Memory used: 26 MB.
01.May.2025 - 14:38:41 Starting FileRemover: 0 Items to delete.
01.May.2025 - 14:38:41 Checking for folder deletion Tasks.
01.May.2025 - 14:38:41 Skipped root folder: D:\VODS\goig3d-test.m3u (contains .strm files or subfolders containing them).
01.May.2025 - 14:38:41 All delete tasks completed.
01.May.2025 - 14:38:41 All Updating tasks completed.
01.May.2025 - 14:38:41 Debugging: Item(s) count: 1
01.May.2025 - 14:38:41 Debugging: StrmsToWriteBag.Count=1
01.May.2025 - 14:38:41 Starting File Writer: 1 Items to write/update.
01.May.2025 - 14:38:41 Item [1] Already Exists! - I/O Queued for Overwwriting [TV Shows]: D:\VODS\goig3d-test.m3u\TV Shows\Mystery\Big Mouth\Season 01\Big_Mouth_S01_E08.strm
01.May.2025 - 14:38:41 Debugging: Fetch Images: False.
01.May.2025 - 14:38:41 Finished this import.
01.May.2025 - 14:38:41 Task Ended at: 01/05/2025 14:38:41
01.May.2025 - 14:38:41 Task Duration: 00:00:01.93

@BigIrn

Finally found a place to put the string manipulations after ignore/includes so they behave as before adding underscores etc.
The extra manipulations here were added when trying to find out what was causing re-writes so not much thought was given to where it should happen, but it seems ok now.
 

Please test away

VODtoSTRMscheduledTask.dll 905.5 kB · 5 downloads

Womp womp... still in genres.

 

embyserver(8).txt ScheduledTask-[M3UToStrm]03.May.2025 - 10.41.08.txt

 

Edited May 4, 2025 by GrimReaper
Attachment removed

[mickle026 650]

9 hours ago, BigIrn said:

Womp womp... still in genres.

embyserver(8).txt 26.85 MB · 0 downloads ScheduledTask-[M3UToStrm]03.May.2025 - 10.41.08.txt 39.25 MB · 0 downloads

 

Are you sure this is not a group-title in your m3u file? the log you posted does not show any of the m3u elements being read so I cannot tell.
but with group-title being unchecked, it should create one so my suspicion is this is group-title in your m3u file and your provider has used genres there.
Try "Don't Create a group title folder" in the screenshot above and see if its the same

Can you send me a copy of your m3u via pm.  Obfuscated is fine.  Select obfuscated output and a type and it'll create it for you in your split strm directory
Screenshot below shows how to do this

 

If you have Extra Verbose logging on, Constructed Folder Name [2]: groupTitle=

if this is a genre then turn on "Don't create a Group Title Folder"

Edited May 4, 2025 by GrimReaper
Edited Quote

[sundevil67 28]

5 hours ago, mickle026 said:

v1.0.0.174

ApolloTV - TV Shows

I have added a retry mechanism, up to 5 times per file
I have added a delay between download attempts - each attempt waits a little longer than the last.
I have changed the url code to download from the number you specify on the end of your url string, if no number is specified then it will start at no.1
I have added a failed concurrency check (failed files in a row)
Timeout is now set at 200 seconds per attempt

currently it will try to download the parts past the number you specify, but after 5 failed attempts it will move on to the next part, so if there are 29 parts it will try part 30 before stopping. Part 30 should be a 404 not found error, it will stop downloading at a 404 error and start the processing.  404 should be there are no more files?
If there a two files in a row that fail 5 attempts to download, it will give up downloading and start processing.  (fall back if there is not a 404 error - but will take 10 attempts and 2.35 mins + timeouts extra to achieve this failure before continuing) - worst case with timeouts is just short of 10 mins extra.

This might increase the download times, but should be a lot more robust.
 

I just shut it down after 12 hours & getting stuck at 26% ... it seemed to go through & download all of them; the log was huge. I'll give this new build a go now & fingers crossed...

VODtoSTRMscheduledTask.dll 938 kB · 0 downloads

 

[Apotropaic 41]

7 hours ago, mickle026 said:

Are you sure this is not a group-title in your m3u file? the log you posted does not show any of the m3u elements being read so I cannot tell.
but with group-title being unchecked, it should create one so my suspicion is this is group-title in your m3u file and your provider has used genres there.
Try "Don't Create a group title folder" in the screenshot above and see if its the same
 

I think @mickle026is right here, they look like the same Group Titles that I have in mine, so are more like categories the provider has set. @BigIrnif you follow what was recommended it will put all the movies into the root of the folder, for me the list is just too big and I prefer to pick and choose which film 'category' I import into my library.

[goin3d 4]

Got past my caching issue and back to test runs!  The new language filters are working perfectly!  In the latest build I'm having trouble out putting the additional files that come in run after run.  For example in this run they way i read it there are 28 items to be written:

04.May.2025 - 09:57:32 -----------------------[ Updating Output Folder ]-------------------------------
04.May.2025 - 09:57:32 Debugging: Remove Obsolete Files: False.
04.May.2025 - 09:57:32 All Updating tasks completed.
04.May.2025 - 09:58:43 Debugging: Item(s) count: 28
04.May.2025 - 09:58:43 Debugging: StrmsToWriteBag.Count=28
04.May.2025 - 09:58:43 Starting File Writer: 28 Items to write/update.
04.May.2025 - 09:58:43 All write tasks completed.
04.May.2025 - 09:58:43 Debugging: Fetch Images: False.
04.May.2025 - 09:58:43 Finished this import.
04.May.2025 - 09:58:43 Task Ended at: 5/4/2025 9:58:43 AM
04.May.2025 - 09:58:43 Task Duration: 00:04:14.54

In the file write log all that i see is:
04.May.2025 - 09:58:43 Initial queue size: 0
 

One example that i see is in the m3u but not being written to disk is attached.  

 

Casualty.txt

[sundevil67 28]

15 hours ago, sundevil67 said:

 

Still can't seem to write anything

ScheduledTask-[M3UToStrm]_03.May.2025-19.54.59.txt.zip

[mickle026 650]

49 minutes ago, sundevil67 said:

Still can't seem to write anything

ScheduledTask-[M3UToStrm]_03.May.2025-19.54.59.txt.zip 3.5 MB · 3 downloads

I can't read the log file properly on my phone at work, but I am glad to see the downloading is working now.

I will look as soon as I am able.  I will be back home tomorrow.

[mickle026 650]

1 hour ago, goin3d said:

Got past my caching issue and back to test runs!  The new language filters are working perfectly!  In the latest build I'm having trouble out putting the additional files that come in run after run.  For example in this run they way i read it there are 28 items to be written:

04.May.2025 - 09:57:32 -----------------------[ Updating Output Folder ]-------------------------------
04.May.2025 - 09:57:32 Debugging: Remove Obsolete Files: False.
04.May.2025 - 09:57:32 All Updating tasks completed.
04.May.2025 - 09:58:43 Debugging: Item(s) count: 28
04.May.2025 - 09:58:43 Debugging: StrmsToWriteBag.Count=28
04.May.2025 - 09:58:43 Starting File Writer: 28 Items to write/update.
04.May.2025 - 09:58:43 All write tasks completed.
04.May.2025 - 09:58:43 Debugging: Fetch Images: False.
04.May.2025 - 09:58:43 Finished this import.
04.May.2025 - 09:58:43 Task Ended at: 5/4/2025 9:58:43 AM
04.May.2025 - 09:58:43 Task Duration: 00:04:14.54

In the file write log all that i see is:
04.May.2025 - 09:58:43 Initial queue size: 0
 

One example that i see is in the m3u but not being written to disk is attached.  

 

Casualty.txt 5.82 kB · 1 download

What did the log say for the io queuing?

There should be at least an entry between starting files written and the write tasks ending, there should be a reason.  I cannot tell feo that log file.

Can you send me your config file and your m3u so I can run the same as you in a debug console?

Again an obfuscated m3u is fine, I just need to see what is or isn't happening.

[goin3d 4]

1 hour ago, mickle026 said:

What did the log say for the io queuing?

There should be at least an entry between starting files written and the write tasks ending, there should be a reason.  I cannot tell feo that log file.

Can you send me your config file and your m3u so I can run the same as you in a debug console?

Again an obfuscated m3u is fine, I just need to see what is or isn't happening.

I've attached an m3u, IO entries from 2 back to back runs, and my config.  the items that show up for IO seem to change with each run even though the m3u contents aren't changing.  Sometimes it says 0 when there should be some.  sometimes it has items that already exist.

IO example.zip

[mickle026 650]

21 hours ago, goin3d said:

I've attached an m3u, IO entries from 2 back to back runs, and my config.  the items that show up for IO seem to change with each run even though the m3u contents aren't changing.  Sometimes it says 0 when there should be some.  sometimes it has items that already exist.

IO example.zip 2.23 MB · 2 downloads

Thanks, I will investigate this and see if I can reproduce it or at least find out what is going on.

first run

Queued    54,719 items were for writing.
Written   54,692 Files, 14,488 Folders (windows explorer)
nexr scan 54,692 strms found on next directory scan (confirmend by plugin)

27 files either failed or were duplicates.



2nd run
05.May.2025 - 17:48:45 [AddToContentListAsync] Adding to m3uContents - Count = 64968
05.May.2025 - 17:48:45 Live TV M3U Items found: 8180.
05.May.2025 - 17:48:45 Movies M3U Items found: 7968.
05.May.2025 - 17:48:45 TV Show M3U Items found: 48820.
05.May.2025 - 17:49:03 Debugging: StrmsToWriteBag.Count=2936

Starting File Writer: 2936 Items to write/update.
Queued       206 items were for writing.
Written   54,903 Files, 14,517 Folders
Found     54,903 files with extension .strm

should have written: 7968 + 48820 = 56,788 items 

this - previous: 54,903 - 54,719 =  184 <-- number doesnt match! and still not 56,788 strms



third run
05.May.2025 - 18:09:27 Found 54903 files with extension .strm
05.May.2025 - 18:30:01 [AddToContentListAsync] Adding to m3uContents - Count = 64968
05.May.2025 - 18:30:01 Live TV M3U Items found: 8180.
05.May.2025 - 18:30:01 Movies M3U Items found: 7968.
05.May.2025 - 18:30:01 TV Show M3U Items found: 48820.

05.May.2025 - 18:30:19 Debugging: StrmsToWriteBag.Count=2612
2936 - 2612 = 324
05.May.2025 - 18:30:20 Queued 195 items were for writing.
05.May.2025 - 18:30:20 Debugging: Duplicate files in m3u (not requeued): 0

 I don't have much time today, but I will update this.....to be continued....

so far it looks like not all files are written on the first run.
subsequent runs are adding what is missing, but not all of them each time

 

The m3u decoding counts are the same each time:

conclusion: something is wrong in the logic that decides whether to write, update them or not 

Edited May 5, 2025 by mickle026