Plugin: Export VODS from .m3u - split m3u livetv, tvshows, movies (Going BETA!!)

[goin3d 4]

updated my previous post to contain the correct log entry!

[mickle026 650]

37 minutes ago, goin3d said:

putting more and more data through now.  it looks like there are still some items not being picked up correctly.  for example:

#EXTINF:-1 tvg-name="Under the Banner of Heaven S01E04" tvg-logo="https://xx.xx:443/images/7zTrdsAHDG9LdY7pQX1aRTuFTerAGqgdq-dRx7OiqKvJmQDH5v2La89R9I4zuWJbkRz_U5U23VuD7hsibuY8fG3iLqSlwMmf6uktCF-qKk0.jpg" group-title="Mystery",Under the Banner of Heaven S01E04
https://xxx.xx:443/series/xxxx/xxxx/2121006.mkv

looks like in the log it's coming back as

Type: Live TV - Did not detect file extension (most likely a continuous stream): .jpg" group-title="crime",under the banner of heaven s01 e04

 

I'll play with the country code changes you just posted today.  What i'm learning is with large amounts of data exactly how we choose to filter out/in the country codes actually makes a big difference in overall processing time.  

can you send me your m3u?

I don't mind if you obfuscate it first, i.e. use a text editor such as notepad and find & replace all such as  wesiteurl.com, username and password.  I am not interested in using any of the urls so don't worry about that, its testing it and taking a look at what it is failing on. 

[mickle026 650]

1 hour ago, goin3d said:

What i'm learning is with large amounts of data exactly how we choose to filter out/in the country codes actually makes a big difference in overall processing time. 

Yes, all manipulations take cpu cycles,  so the more cycles, the more cpu time, the less logical cpus and threads available the more noticeable it is.

if for example you just include starts with: "EN -" & remove country code just "EN" then you have removed lots of cycles from each item.

195 countries & 14 variants = 2730 manipulations per string to remove 1 country code (approximately 38,220 cpu cycles) 0.597 milliseconds on my machine (0.000597 seconds)

1 country and 14 variant = 14 manipulations to remove 1 country code (approximately 196 cpu cycles) - 49 nano seconds on my machine (0.000000049 seconds)

So 1million entries in an m3u,
Removing country code on a 4ghz 16 logical cpu computer:

1 code - 0.49 seconds (less than a minute) - a user override list over 1 million entries (almost nothing) - 196,000,000 cpu cycles
195  codes - 597 seconds (9 minutes and 57 seconds) - the built in list over 1 million entries (almost 10 minutes) - 38,220,000,000 cpu cycles

Apply this logic to all the manipulations, includes, excludes etc
There are 14 items per country code with the variants built into the plugin as opposed to only 1 per ignore etc,
so 1 ignore on my machine is 49 nano seconds / 14 = 3.5 nanoseconds per 1 million items (not even noticeable)

 

It is mostly the log writing that uses the time, but without it you cannot see what is going on....

Edited April 19, 2025 by mickle026

[mickle026 650]

Repost of Earlier build with save custom country code fix (and ignore contains)

Apologies, schoolboy error .......
 

 

 

 

VODtoSTRMscheduledTask.dll

Edited April 19, 2025 by mickle026

[Jekaboom 4]

Hi mickle026 - just want to leave a big thank you - this works extremly well and is much more convenient compared to all the other M3U to STRM options I have found. This is an essential plugin for anyone that want to use Emby for M3U based VOD. I found the best combination is to use the built in IPTV plugin for Live TV, and this plugin for anything VOD related. 

Looking forward to any future enhancements - but it does the basic job already pretty well

[mickle026 650]

4 minutes ago, Jekaboom said:

Hi mickle026 - just want to leave a big thank you - this works extremly well and is much more convenient compared to all the other M3U to STRM options I have found. This is an essential plugin for anyone that want to use Emby for M3U based VOD. I found the best combination is to use the built in IPTV plugin for Live TV, and this plugin for anything VOD related. 

Looking forward to any future enhancements - but it does the basic job already pretty well

Thanks, did you notice this plugin also splits the m3u for you with only the entries you want , so you can have only live tv in your live tv tuner m3u file?

[Jekaboom 4]

2 minutes ago, mickle026 said:

Thanks, did you notice this plugin also splits the m3u for you with only the entries you want , so you can have only live tv in your live tv tuner m3u file?

Yes, thank you for pointing this out. I use a custom M3U that only has VOD as input and a seperate M3U with only live tvcontent for the tuner plugin, works really well

[mickle026 650]

This is @gillmacca01's m3u file, one I have saved on disk, not a newly updated one

19.Apr.2025 - 19:42:33 CPU Cores: 8: Logical Processors (Max Threads): 16, Hyperthreading: True
19.Apr.2025 - 19:42:33 Using max 16 threads
19.Apr.2025 - 19:42:37 Total m3u Items: 590608
19.Apr.2025 - 19:42:38 Low memory mode enabled. RAM threshold set to: 3276.80 MB
19.Apr.2025 - 19:49:11 Task Ended at: 19/04/2025 19:49:11
19.Apr.2025 - 19:49:11 Task Duration: 00:06:37.25
19.Apr.2025 - 19:49:11 Finished.

It also scanned the contents of the output directory that contained a previous run of someone else's created files & deleted them i this time with simplified logging on (not verbose)

[goin3d 4]

1 hour ago, mickle026 said:

Repost of Earlier build with save custom country code fix (and ignore contains)

Apologies, schoolboy error .......
 

 

 

 

VODtoSTRMscheduledTask.dll 873.5 kB · 0 downloads

Sorry, you lost me, is this one the latest build?

[mickle026 650]

2 minutes ago, goin3d said:

Sorry, you lost me, is this one the latest build?

Yeah, the last one wasn't saving the custom country codes.  Without saving it's not available to the server side where the code runs, it would still use the built-in list.

Not sure if I increased the build number, but yes it's the latest build.

[goin3d 4]

1 hour ago, mickle026 said:

Yeah, the last one wasn't saving the custom country codes.  Without saving it's not available to the server side where the code runs, it would still use the built-in list.

Not sure if I increased the build number, but yes it's the latest build.

got it ty!!! just finished a very large run and here's what i've noticed.

1.  after using the most current version you posted i didn't see any titles that weren't identified as i posted.

2.  this might be related to the file system i run on my NAS, but any title with a ':' in it gets destroyed.  for example the directory and files get created as something like E0O3E0~W.  i'm assuming what i need to do is add : to the remove filter?

3.  when processing multiple m3u files i had one run where the first file failed to download.  if this occurs it looks like the whole stop stops rather than going to the next feed.  Is that what you expected?

[mickle026 650]

29 minutes ago, goin3d said:

got it ty!!! just finished a very large run and here's what i've noticed.

1.  after using the most current version you posted i didn't see any titles that weren't identified as i posted.

2.  this might be related to the file system i run on my NAS, but any title with a ':' in it gets destroyed.  for example the directory and files get created as something like E0O3E0~W.  i'm assuming what i need to do is add : to the remove filter?

3.  when processing multiple m3u files i had one run where the first file failed to download.  if this occurs it looks like the whole stop stops rather than going to the next feed.  Is that what you expected?

Does the Nas support long filenames? 

This has come up several times from the start of this thread where directories appear as different shortened versions.  It can either r be the Nas or support is off in your os.  In windows there is a registry setting to turn long filename support on when viewing in explorer. 

This is the reason for being able to exclude group title and season folders so you can shorten the path structure.

 

I will have to relook at the procedure for multiple downloads and how to handle that.

[goin3d 4]

3 minutes ago, mickle026 said:

Does the Nas support long filenames? 

This has come up several times from the start of this thread where directories appear as different shortened versions.  It can either r be the Nas or support is off in your os.  In windows there is a registry setting to turn long filename support on when viewing in explorer. 

This is the reason for being able to exclude group title and season folders so you can shorten the path structure.

 

I will have to relook at the procedure for multiple downloads and how to handle that.

Thanks, i did verify it wasn't length, but the : character itself.  I just didn't know given it is sometimes a special character if it would throw off something else if trying to filter it out.

[mickle026 650]

9 minutes ago, goin3d said:

Thanks, i did verify it wasn't length, but the : character itself.  I just didn't know given it is sometimes a special character if it would throw off something else if trying to filter it out.

These should already be filtered out of the strm files and folders.

Any chance of sending me the affected m3u?, like I said earlier I don't mind if you obfuscate password, usernames and URLs.  It helps me debug different types of m3u structures and problems.

[goin3d 4]

26 minutes ago, mickle026 said:

These should already be filtered out of the strm files and folders.

Any chance of sending me the affected m3u?, like I said earlier I don't mind if you obfuscate password, usernames and URLs.  It helps me debug different types of m3u structures and problems.

sent ya a quick test file with the : item and the EN| country code.  Putting the : into the filter box did work, but it wasn't happening by default.  I'm also trying to figure out a way to remove the EN| from files without turning on the country code filter.  The challenge is the pipe | is automatically being removed leaving "EN ".  I can't seem to get a space to filter out using the remove string option.

[sundevil67 28]

I don’t want to add LiveTV channels to that same directory where I’m storing VOD movies & TV, but am wondering if my issues (nothing before 2023 & more recent shows missing) might be resolved by adding a 3rd URL. I’m wondering this since noticing some fine print below this URL on their website that says:  “includes Channels list and recent VOD”. 

It is the same URL as the others, but instead of ending in “movies” or “tvshows”, it ends with “livetv”. Unfortunately, when I pop that link into a browser, I get the following message:

{"success":false,"message":"Resource not found"}

I’m wondering if anyone else is running into the same thing. "Useless" isn't a strong enough word to describe their support.

[mickle026 650]

1 hour ago, goin3d said:

sent ya a quick test file with the : item and the EN| country code.  Putting the : into the filter box did work, but it wasn't happening by default.  I'm also trying to figure out a way to remove the EN| from files without turning on the country code filter.  The challenge is the pipe | is automatically being removed leaving "EN ".  I can't seem to get a space to filter out using the remove string option.

Absolutely no issues whatsoever with that file

 

ScheduledTask-[M3UToStrm]_20.Apr.2025-00.59.29.txt

M3u is 521 lines

1st line is not data, others are 2 lines per entry = 260 items.  It wrote 260 strm files.

Ran again with "EN,[EN]" as custom country codes to remove

 

So I think these items are not what is causing your issue

 

#EXTINF:-1 tvg-id="" tvg-name="EN| American Dragon: Jake Long S02 E07" tvg-logo="http://xxxxxx:900/images/3bKvoRm9JuFPmi2YvPjA7qPKOvM_small.jpg" group-title="[EN] KIDS",EN| American Dragon: Jake Long S02 E07

It is more likely an extra line in your m3u  probably starting #EXT- something or other..

Edited April 20, 2025 by mickle026

[mickle026 650]

1 hour ago, sundevil67 said:

I don’t want to add LiveTV channels to that same directory where I’m storing VOD movies & TV, but am wondering if my issues (nothing before 2023 & more recent shows missing) might be resolved by adding a 3rd URL. I’m wondering this since noticing some fine print below this URL on their website that says:  “includes Channels list and recent VOD”. 

It is the same URL as the others, but instead of ending in “movies” or “tvshows”, it ends with “livetv”. Unfortunately, when I pop that link into a browser, I get the following message:

{"success":false,"message":"Resource not found"}

I’m wondering if anyone else is running into the same thing. "Useless" isn't a strong enough word to describe their support.

I am very sorry, I really do not understand this post & what it is your are asking....

 

Doesn't all your channels (livetv, movies, tvshows) come in the same m3u file?

Quote

 “includes Channels list and recent VOD”

This suggests that it does, and you only need 1 m3u file

Edited April 20, 2025 by mickle026

[goin3d 4]

Glad you were able to run them!  what's weird about the : is that ii is fine on my windows machine and gets automatically taken out.  on my NAS with a different file system it does not and creates bad filenames unless i add : to the filter.  i'll just remember that one.

About the EN| entries.  can you think of a way to remove them without using the country code logic?  In order to speed it up i was trying to take care of that single removal though the custom strings logic, but the | gets automatically removed and the EN is left with a space at the end which i haven't figured out how to remove (or if i can) via the filter.  maybe the country code entry is the only way?

Thank you so much looking into all these!  hopefully others will find these useful as well.

[mickle026 650]

5 minutes ago, goin3d said:

Glad you were able to run them!  what's weird about the : is that ii is fine on my windows machine and gets automatically taken out.  on my NAS with a different file system it does not and creates bad filenames unless i add : to the filter.  i'll just remember that one.

About the EN| entries.  can you think of a way to remove them without using the country code logic?  In order to speed it up i was trying to take care of that single removal though the custom strings logic, but the | gets automatically removed and the EN is left with a space at the end which i haven't figured out how to remove (or if i can) via the filter.  maybe the country code entry is the only way?

Thank you so much looking into all these!  hopefully others will find these useful as well.

Did you try it in the latest code?
I added an over ride filter so that it doesn't use the built-in ones
The text Area below is new to the last build

 

So when I ask the filesystem for unsupported file-system characters, the NAS cannot be reporting them????

urgh - that means you probably need a box for that as-well   - jeez!

 

Edited April 20, 2025 by mickle026

[goin3d 4]

3 minutes ago, mickle026 said:

Did you try it in the latest code?
I added an over ride filter so that it doesn't use the built-in ones
The text Area below is new to the last build

Yes, trying that exactly as you have it now and timing out the total run.

One more thing on the multi-file/file failing item that i'm seeing.  I have "Clean Out/Remove Old (obsolete) entries from the output directory." checked to keep the output folder current.  If for any reason 1, 2, or 3 of the m3u's i enter can't connect, then the whole run fails AND the output folder is being deleted.  If something fails should that happen?  

[mickle026 650]

30 minutes ago, goin3d said:

Yes, trying that exactly as you have it now and timing out the total run.

One more thing on the multi-file/file failing item that i'm seeing.  I have "Clean Out/Remove Old (obsolete) entries from the output directory." checked to keep the output folder current.  If for any reason 1, 2, or 3 of the m3u's i enter can't connect, then the whole run fails AND the output folder is being deleted.  If something fails should that happen?  

I still have to look at download failures in the input boxes and how to handle that, but I have changed the logic for illegal chars to cover all filesystems, and use a fallback if it cannot fetch them
 

 char[] InvlaidPathChars = Path.GetInvalidPathChars();
 char[] InvalidFilenameChars = Path.GetInvalidFileNameChars();

 // Combine both arrays and remove duplicates
 char[] illegalChars = invalidFileNameChars
                        .Concat(invalidPathChars)
                        .Distinct()
                        .ToArray();

 if (illegalChars.Length == 0)
 {
     illegalChars = {
         '<', '>', ':', '"', '/', '\\', '|', '?', '*', // Windows
         '/', '\0',                                   // Linux/Unix
         ':', '\0'                                    // macOS
     };
 }

 // Add ASCII 0–31 (control characters) to the list
 char[] asciiControlChars = Enumerable.Range(0, 32)
                                      .Select(x => (char)x)
                                      .ToArray();

 // Combine all invalid characters and remove duplicates
 char[] illegalPathChars = invalidChars
                               .Concat(asciiControlChars)
                               .Distinct()
                               .ToArray();
}

See if this build resolves your issue with that first  

see next posts

 

Edited April 20, 2025 by mickle026

[goin3d 4]

You’re awesome!  I’ll give it a try

[mickle026 650]

Revised it again, if you downloaded it please re-download it

[mickle026 650]

The entry at the top of the log should now contain characters if it was empty, it might vary depending on the system, but it should look similar

 

VODtoSTRMscheduledTask.dll

 

so tired last night that i deleted the wrong one from the posts - i'm only human after all ..... reattached it.

 

Edited April 20, 2025 by mickle026