Jump to content

python script to convert m3u file to strm files with emby folder structure (free)


t1m

Recommended Posts

gillmacca01

I have the m3u files already downloaded, but do not know what to edit in the bat file.

Sorry about this, probably simple for someone like you

Link to comment
Share on other sites

hooray4me
10 hours ago, cayars said:

 

 

I was curious to see if any of you tackled it in an elegant way.  Last year was I was playing with this I had 6 or 7 IPTV providers I could generate streams from. That's on top of 20K+ movies and 100K+ episodes in my libraries.

Ideally this could be found out using the Emby API with a call for each movie/episode but that would be a lot of hits. This is a one-off for my system but what I did was Generate the strm files including normal folder paths naming. 2 of the providers had pretty bad naming of movies so results were always questionable. So I'd generate everything based on the naming in the m3u files. Each provider had it's own local working space. I set this up so it didn't overwrite files that already exist.  That allows me to then pull only files with a newer timestamp for further processing.  Anything new would get ran through a file renamer util so regardless of source would have the same naming. I'd then load them by preference of provider.  First run would load everything.  I'd then scan it into Emby along with my other media.  Once done loading I'd shut down, grab a copy of the library.db and restart Emby.

With some SQL I  could then find dupes IDs and would write these out to a text file file.  The text file was actually a batch file that when ran would delete the strm files getting rid of dupes.  I wrote a little app that would remove folders that were empty to keep things a bit more tidy.  I'd just work down the line until I've loaded everything and deduped it.  A bit of work but it worked quite nicely.

Round 2.  Instead of processing files separately I setup a database and imported each m3u file including tv channels and vod content. With everything in one DB I made a small util that would test each channel and VOD channel and record the resolution, bitrates, audio languages and other similar info I could get.

With all providers and channels in the DB I had to handle renaming and identification of movies and shows in the db vs file system but wasn't hard. I could cleanup TV channel names as well using a standardized naming convention.  I could then filter out channels I have via OTA or cable. So now with 3 10 or 50 providers the job isn't really harder.

Besides the information I 've loaded from the m3u files I've also imported list of movies and episodes from my Emby system which I'd import roughly once a month.

 

This added additional features as well.  I could for example do everything as mentioned above with the export tailored to Emby Server use.  I could also ignore or forget I have any of my own content and just get a combined deduped list of all channels and vod back out as one m3u file which could be used conventionally from a phone or other device. My daughter for example does a lot of traveling and takes a small MiBox (vs Amazon Firestick) with her. Now anywhere with Internet she has a really nice collection of VOD as well as full channel list of TV channels by way of the Live Channels app (not available I don't think on Fire TV) which does it's own guide loading so one less thing to worry about.

Back to Emby...

Few of you would need anything as elaborate and probably have a single provider or two.  I've not tried this yet but imagine it would work well and that is using Chef's version of Auto-Organize to load the VOD content.  This should be able to filter out existing content so it doesn't get loaded. First pass might take a while to load but after the first export/import into Emby you can now process only new content  moving forward.  That assumes you setup your m3u2strm export to not overwrite existing files. You simply grab content with new file timestamps for processing which is easy to do on Windows and Linux.

Which can identify items that exist already.

Then as a backup you could also try the following plugin that combines "duplicate" versions of movies (caution if you want 4K and 1080 separate). Emby should handle merging TV shows on it's own.

So that's the different options I've played with. Having the custom database with the ability to validate and pick the best version (resolution, bitrate) does expand abilities a lot especilally if selection and export is automated like I had it.  Any one provider having issues or dead channels/vod would be found and not used.

The ideal way to do that is using a "redirection" plugin that Emby calls.  So you feed Emby "fake URLs pointing to your own app". The apps get the vod or channel number, dynamically pulls the best real URL to use and returns it to Emby as a redirect which Emby follows.   So the app needs to run but isn't involved in the transport of the thread.  It's just a traffic cop. Since it's testing channels throughout the day it can note when channels are down or find resolution changes, etc and deal with them real-time as Emby is about to use the stream.

With a slight modification the backend program could poll Emby via API to see what channels/streams are in use and act on that info.  If for example you can only get 2 streams per subscription you could have 2 or 3 subs from the same provider with different username/passwords. The backend would know 2 streams are in use from the provider so any new use would use the stream (username,passcode) from the 2 sub...  Poor man's m3u bonding if you want to call it that. But unlike XTeVe it's not sitting in the middle processing the stream just doing some quick traffic direction at the start of stream use which basically uses no additional system resources in a meaningful way.

Carlo

 

I'm going to have to read this about 3 times and get back to you lol. I really like where you're going with it.

Edited by hooray4me
Link to comment
Share on other sites

afogleson
4 hours ago, gillmacca01 said:

I have the m3u files already downloaded, but do not know what to edit in the bat file.

Sorry about this, probably simple for someone like you

This is where i think it  is better to create a file of urls but you can do this you just have to have it run 3 times (once for each M3U) the command would be:

CreateStreams.bat -i false -n pathToFile1 -o path-to-where-you-want-to-save-the-stream-files -u true (if you want a separate folder for each tv show or movies

 

Link to comment
Share on other sites

gillmacca01

Thanks. I'm at work till late tonight, but will give it a try in the morning 

Link to comment
Share on other sites

gillmacca01

Getting somwhere now.

Used CMD: CreateStreams.bat -i false -n Y:\bin\event.m3u -o Y:\bin\VOD Sports -u true

I have tried running from the downloaded event file and also tried added the 3 links to a file, but getting the following message:

Handled 0 entries

I have confirmed the downloaded file has links

 

 

Link to comment
Share on other sites

afogleson
7 hours ago, gillmacca01 said:

Getting somwhere now.

Used CMD: CreateStreams.bat -i false -n Y:\bin\event.m3u -o Y:\bin\VOD Sports -u true

I have tried running from the downloaded event file and also tried added the 3 links to a file, but getting the following message:

Handled 0 entries

I have confirmed the downloaded file has links

 

 

ahhh so im guessing here but java does not handle well the spaces so i am betting you need to probably put "Y:\bin\VOD Sports"   in quotes at best (which i doubt will work right now but i can make work in the next release) or rename the file without spaces

Link to comment
Share on other sites

gillmacca01

I removed the spaces, and it started running, but have run into 2 further issues, A resolution for the issue might resolve the second issue.

Firstly I'm running java 18, in case this helps

1st issue; I am getting java errors in the cmd console (ignore 'Handled 0 entries', as this had a figure last time, and there is nothing new to import)

Y:\bin>CreateStreams.bat -i false -n Y:\bin\events.m3u -o Y:\bin\VODSports -u true
java.nio.file.InvalidPathException: Trailing char < > at index 24: Y:\bin\VODSports\MLB\ NL
        at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:191)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
        at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:232)
        at java.base/java.nio.file.Path.of(Path.java:147)
        at java.base/java.nio.file.Paths.get(Paths.java:69)
        at com.afogleson.m3u.parser.M3UFileParser.createTvFile(M3UFileParser.java:211)
        at com.afogleson.m3u.parser.M3UFileParser.handleEntry(M3UFileParser.java:186)
        at com.afogleson.m3u.parser.M3UFileParser.handleEntries(M3UFileParser.java:121)
        at com.afogleson.m3u.parser.M3UFileParser.parse(M3UFileParser.java:109)
        at M3uHandler.createAndRun(M3uHandler.java:78)
        at M3uHandler.main(M3uHandler.java:18)
Handled 0 entries.

The 2nd issue - it is creating folders with content, but the type is file, not strm (Emby cannot import these)

image.thumb.png.21bb8d743d369df09d2fc3b224233a0c.png

 

Also seem to be missing whole folders. E.g. WWE, but again could be due to the java error

Edited by gillmacca01
Link to comment
Share on other sites

afogleson
3 hours ago, gillmacca01 said:

Any help with this please

I got bogged down with my day job and have not had a chance to even glance at this. which VOD urls are you using (I presume Apollo since thats all i have to test with and that i had it working with before. but if it is someone else i would need a (sanitized) version of their M3U to test with. the problem i had was not with creating the strm files or such but in having emby show the saved files (i can remember what media type i said it was... tv or movie but it worked but still looked a little strange)

Link to comment
Share on other sites

gillmacca01

I'm using apollo, and all 3 event m3u files. 

I get other java errors with the event_2 and event_3, which I believe maybe due to the space issue

Link to comment
Share on other sites

afogleson
10 hours ago, gillmacca01 said:

I'm using apollo, and all 3 event m3u files. 

I get other java errors with the event_2 and event_3, which I believe maybe due to the space issue

OK i have just tested this locally...

So just in case there is some difference between what i have locally and what you have i have attached the program again.

I have also attached the file i used to produce my stream files (of course you would need to replace your email / password but you can see i am using all 3 streams also)

and a picture of the options i selected on the UI to do this. (please note that NOT all of the options are available in even the help menu as some of those are features I am working on a tthe moment.

but if i do all that i get a LOT of  stream files :)

 

Finally if using from the command line here is what i call to do this (using the same setup as in the ui picture)  and i get 10033 streams created

CreateStreams-1.0/bin/CreateStreams -i false -n /home/afogleson/Videos/SportVOD.m3u -o /home/afogleson/vod -u true
 

The interesting thing to note about the way that apollo does Sport VOD is that using -u true or false changes nothing. it always will group them into folders by event type. but everything does seem to be working

createStreamsUI.png

streamFiles.png

CreateStreams-1.0.zip SportVodExample.m3u

  • Like 1
Link to comment
Share on other sites

gillmacca01

hmmmm....interesting.

I was missing a couple of jar files, and now things have improved, but a couple of things.

Your saying your get 10033 streams, I'm only getting 9963 (strm files are now being created successfully)

Still looks like I am missing some sport categories: WWE, UFC, UEFA &  UFC FN - IGNORE THIS, LOOKS LIKE THERE ARE MORE M3U FILES NOW

Never seen the M3U Handler window (although I don't think I need it)

Still getting java errors - I have attached a txt file with the java error messages I am getting from the CMD window

javaerror.txt

Edited by gillmacca01
Link to comment
Share on other sites

afogleson
Just now, gillmacca01 said:

hmmmm....interesting.

I was missing a couple of jar files, and now things have improved, but a couple of things.

Your saying your get 10033 streams, I'm only getting 9963 (strm files are now being created successfully)

Still looks like I am missing some sport categories: WWE, UFC, UEFA &  UFC FN

Never seen the M3U Handler window (although I don't think I need it)

Still getting java errors - I have attached a txt file with the java error messages I am getting from the CMD window

javaerror.txt 52.12 kB · 0 downloads

if you start the program with NO options it will show the UI screen. weird that I dont see the errors on pipe characters. of course a pipe char is a valid character on a linux system.  but i suspect that is why you do not get 10033 entries Let me look at my clean path function and see if i can fix that quick

Link to comment
Share on other sites

afogleson
6 minutes ago, gillmacca01 said:

hmmmm....interesting.

I was missing a couple of jar files, and now things have improved, but a couple of things.

Your saying your get 10033 streams, I'm only getting 9963 (strm files are now being created successfully)

Still looks like I am missing some sport categories: WWE, UFC, UEFA &  UFC FN - IGNORE THIS, LOOKS LIKE THERE ARE MORE M3U FILES NOW

Never seen the M3U Handler window (although I don't think I need it)

Still getting java errors - I have attached a txt file with the java error messages I am getting from the CMD window

javaerror.txt 52.12 kB · 1 download

Ok here you go a version that handles the pipes.

 

CreateStreams-1.0.zip

  • Like 1
Link to comment
Share on other sites

afogleson
3 hours ago, gillmacca01 said:

Might be related to the java error, but the WWE folder is populating with just 2 strm. Looking at the m3u file, there are loads for WWE

The link is: https://tvnow.best/api/list/email address/password/m3u8/events/4

Yeah not sure what the heck apollo is doing these days... but this seems to clear it up (I made assumptions that things would be valid :) they are not so this double checks things. I get 12.550 (if i remember right) streams with this

 

 

 

CreateStreams-1.0.zip

  • Thanks 1
Link to comment
Share on other sites

gillmacca01

All working now, currently got 12459 strm's, but still importing.

Many thanks for your help

Link to comment
Share on other sites

afogleson
15 hours ago, gillmacca01 said:

All working now, currently got 12459 strm's, but still importing.

Many thanks for your help

Good to hear.

Link to comment
Share on other sites

  • 3 weeks later...
afogleson
On 8/15/2022 at 5:54 PM, sNor4pro said:

If i open CreateStream.bat it closes immediately installed fresh JRE vand JDK without errors.

Can someone tell me why?

Do you get some kind of stack trace?

Link to comment
Share on other sites

kkkiiiikkk

Hey all,

I'm somewhat new to this and you've all worked and collaborate so well here.

@afogleson I am new to python and as an Apollo user, your script caught my attention. It looks like this will solve many of my headaches recently. Could you please tell me how I can get to the GUIs? I think I can figure it out from there.

Link to comment
Share on other sites

afogleson
19 hours ago, kkkiiiikkk said:

Hey all,

I'm somewhat new to this and you've all worked and collaborate so well here.

@afogleson I am new to python and as an Apollo user, your script caught my attention. It looks like this will solve many of my headaches recently. Could you please tell me how I can get to the GUIs? I think I can figure it out from there.

Sure, using the create streams zip you unzip it somewhere. it supports windows, mac and linux so its pretty easy.

to use the gui you would just double click one of the scripts in the bin directory. Since that will call the executable with no parameters it will show the GUI.

Link to comment
Share on other sites

  • 4 weeks later...

Got it working now with correct java path in environment.

Next problem is it doesn't extract the VODs.

It's running for 10+ mins but nothing in my output folder.

My m3u is 28 MBs with 222.836 lines. (it has iptv and vod included)

The TV Shows are formatted like this:

#EXTINF:-1 tvg-name="Evil S01E12" tvg-logo="TMDB LOGO URL" group-title="DE • Drama",Evil S01E12
http://IPTVURL.308054.mkv
#EXTINF:-1 tvg-name="Evil S01E13" tvg-logo="MDB LOGO URL" group-title="DE • Drama",Evil S01E13
http://IPTVURL.308055.mkv
#EXTINF:-1 tvg-name="House of the Dragon S01E03" tvg-logo="TMDB LOGO URL" group-title="DE • Top Aktuell - 2022",House of the Dragon S01E03
http://IPTVURL.308356.mkv

Is something wrong in this format?

I tried it via CLI and GUI no errors shown.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...