Jump to content


Photo

EPG seriously flawed - need to fix mappings


  • Please log in to reply
44 replies to this topic

#1 IkeTaylor11 OFFLINE  

IkeTaylor11

    Advanced Member

  • Members
  • 393 posts
  • Local time: 03:30 AM

Posted 03 March 2019 - 10:22 PM

Ok so today I decided I would add more than just US channels from my IPTV provider and add UK channels as well. Currently I use xtream-editor to fully edit my m3u file so every channel has a unique tvg-id, tvg-name and channel name. This should allow for each channel to only be mapped to its correct EPG from an xmltv file or from Emby Guide Data. However, that is not the case. It seems that Emby is auto mapping channels to EPG however it wants to. There is no set standard for how to match channel to EPG. 

 

Here is an example m3u:

 

#EXTM3U
#EXTINF:-1 tvg-id="1001 BBC ONE" tvg-name="1001 BBC ONE" tvg-logo="https://themediaboxx...05/BBC-ONE.png"group-title="Emby UK",1001 BBC ONE
#EXTINF:-1 tvg-id="1002 BBC TWO" tvg-name="1002 BBC TWO" tvg-logo="https://upload.wikim...BC_Two.svg.png"group-title="Emby UK",1002 BBC TWO
#EXTINF:-1 tvg-id="1003 ITV 1" tvg-name="1003 ITV 1" tvg-logo="https://themediaboxx...18/05/itv1.png"group-title="Emby UK",1003 ITV 1
#EXTINF:-1 tvg-id="1004 CHANNEL 4" tvg-name="1004 CHANNEL 4" tvg-logo="https://themediaboxx...5/Channel4.png"group-title="Emby UK",1004 CHANNEL 4
#EXTINF:-1 tvg-id="1005 CHANNEL 5" tvg-name="1005 CHANNEL 5" tvg-logo="https://themediaboxx..._2016.svg_.png"group-title="Emby UK",1005 CHANNEL 5
#EXTINF:-1 tvg-id="1006 BBC FOUR" tvg-name="1006 BBC FOUR" tvg-logo="https://upload.wikim...C_Four.svg.png"group-title="Emby UK",1006 BBC FOUR
#EXTINF:-1 tvg-id="1007 ITV 2" tvg-name="1007 ITV 2" tvg-logo="https://themediaboxx...18/05/itv2.png"group-title="Emby UK",1007 ITV 2
#EXTINF:-1 tvg-id="1008 ITV 3" tvg-name="1008 ITV 3" tvg-logo="https://themediaboxx...8/05/itv-3.png"group-title="Emby UK",1008 ITV 3
#EXTINF:-1 tvg-id="1009 ITV 4" tvg-name="1009 ITV 4" tvg-logo="https://themediaboxx...8/05/itv-4.png"group-title="Emby UK",1009 ITV 4
#EXTINF:-1 tvg-id="1010 SKY ONE" tvg-name="1010 SKY ONE" tvg-logo="https://themediaboxx...e-OCT-2017.png"group-title="Emby UK",1010 SKY ONE
#EXTINF:-1 tvg-id="1011 SKY ATLANTIC" tvg-name="1011 SKY ATLANTIC" tvg-logo="https://themediaboxx...y_atlantic.jpg"group-title="Emby UK",1011 SKY ATLANTIC
#EXTINF:-1 tvg-id="I4.21234.zap2it.com" tvg-name="I4.21234.zap2it.com" tvg-logo="https://zap2it.tmsim...8711_h3_aa.png"group-title="Local Channels",004 WCCODT
#EXTINF:-1 tvg-id="I5.21236.zap2it.com" tvg-name="I5.21236.zap2it.com" tvg-logo="https://zap2it.tmsim...8708_h3_aa.png"group-title="Local Channels",005 KSTPDT
#EXTINF:-1 tvg-id="I9.24504.zap2it.com" tvg-name="I9.24504.zap2it.com" tvg-logo="https://zap2it.tmsim...8719_h3_aa.png"group-title="Local Channels",009 KMSPDT
#EXTINF:-1 tvg-id="I11.20505.zap2it.com" tvg-name="I11.20505.zap2it.com" tvg-logo="https://zap2it.tmsim...8717_h3_aa.png"group-title="Local Channels",011 KAREDT
#EXTINF:-1 tvg-id="I29.21235.zap2it.com" tvg-name="I29.21235.zap2it.com" tvg-logo="https://zap2it.tmsim...1235_h3_aa.png"group-title="Local Channels",029 WFTCDT
 
 
As you can see in my m3u, none of my channels have any overlapping identifiers. But, epg.png is how Emby is choosing to map my channels. This was done automatically without me mapping them. This is from a UK Emby Guide Source (GBR-1000197-DEFAULT is the Emby Guide Source ID) and channels are mapping to my US zap2it channel names. From what it seems Emby is only using channel number to do the epg matching. This might not be an issue if it took the order of the xmltv files and emby guide data sources into account, but that is not happening either. 
 
What ends up happening is I am not able to assign the correct mapping to the correct channel. I use a mixture of xmltv and Emby Guide Data for my mapping. There needs to be a way for Emby to do strict matching on a particular tag. Choose tvg-id or tvg-name and only match on that. Channel number is not a valid option when doing mappings. 
 
Furthermore once a channel maps there is no way to unmap that channel. This might help with the incorrect mappings since I could go back and reverse the auto mappings Emby did, but I don't see a way to do this. 
 
TO SUMMARIZE: We need a way to know exactly what data is being pulled from Emby Guide Data (an xmltv file would be nice), we need strict channel mapping on either tvg-id or tvg-name (NOT CHANNEL NUMBER), we also need a way to trim down the guide channels being pulled from Emby Guide Data (ability to set favorites and trim non-needed guide data)
 
 

Attached Files

  • Attached File  epg.PNG   174.3KB   8 downloads


#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138075 posts
  • Local time: 04:30 AM

Posted 03 March 2019 - 11:52 PM

Hi, yes we plan to improve our channel management features in an upcoming update to allow you to better control this. Thanks.



#3 Spaceboy OFFLINE  

Spaceboy

    Advanced Member

  • Members
  • 3891 posts
  • Local time: 09:30 AM

Posted 04 March 2019 - 07:00 AM

Reported, I would say, years ago.

First emby was poor at mapping then it got far too enthusiastic, which combined with the inability to unmap made it worse.

Glad that you are catching up on the last couple of years ike .

Still it’s March now, those channel management features can’t be far away . That would probably have been better 4 weeks today!
  • denz and Senna like this

#4 CBers OFFLINE  

CBers

    Advanced Member

  • Moderators
  • 15014 posts
  • Local time: 09:30 AM
  • LocationKent, England.

Posted 04 March 2019 - 08:16 AM

This is why people are starting to use xTeVe.

Once Emby has the channel management, then we can test it out and start using it.

The Emby team is working as hard and as fast as it can to make everyone as happy as possible. Thanks !



#5 Senna OFFLINE  

Senna

    Advanced Member

  • Members
  • 1133 posts
  • Local time: 10:30 AM

Posted 04 March 2019 - 09:07 AM

Ok so today I decided I would add more than just US channels from my IPTV provider and add UK channels as well. Currently I use xtream-editor to fully edit my m3u file so every channel has a unique tvg-id, tvg-name and channel name. This should allow for each channel to only be mapped to its correct EPG from an xmltv file or from Emby Guide Data. However, that is not the case. It seems that Emby is auto mapping channels to EPG however it wants to. There is no set standard for how to match channel to EPG. 
...
...
There needs to be a way for Emby to do strict matching on a particular tag. Choose tvg-id or tvg-name and only match on that.
...
...
Furthermore once a channel maps there is no way to unmap that channel. This might help with the incorrect mappings since I could go back and reverse the auto mappings Emby did, but I don't see a way to do this.
...
...
TO SUMMARIZE: We need a way to know exactly what data is being pulled from Emby Guide Data (an xmltv file would be nice), we need strict channel mapping on either tvg-id or tvg-name (NOT CHANNEL NUMBER), we also need a way to trim down the guide channels being pulled from Emby Guide Data (ability to set favorites and trim non-needed guide data)

Welcome to the Emby Club :rolleyes:

All your points are valid, but also already very intensively discussed in topics below:
Live TV Guide Changes in Emby Server 3.6
New Emby guide provider
 
Currently, there is no list from Emby Guide Data, to use for IPTV users, so they can prepare an m3u with the right tvg-id's to match with Emby Guide data Gracenote ID's as maegibbons already requested in the very early stages of Emby Guide Data development: https://emby.media/c...36/#entry655761
 
Devs are already aware of all of your issues, which for now, causes headache for IPTV users who want to use Emby Guide data and better EPG/channel management.
The conclusion of above topics was, that we just have to wait for the Devs to finalize the new Emby EPG/channel management and hopefully integrate most of the users request found in those two topics, to clear the current headache IPTV users have now with Emby. And if we see some important bits missing, we just remind them again about those ;)


  • Spaceboy, cayars and IkeTaylor11 like this

#6 IkeTaylor11 OFFLINE  

IkeTaylor11

    Advanced Member

  • Members
  • 393 posts
  • Local time: 03:30 AM

Posted 04 March 2019 - 09:38 AM

Hi, yes we plan to improve our channel management features in an upcoming update to allow you to better control this. Thanks.

At this point its not even channel management, its the mapping process that is the biggest issue. While channel management, xmltv files from GN and tag matching would be extremely great, the main issue is mapping. If you could disable auto mapping and allow for unmapping of channels that would resolve all of the issues in the short term. It would make more work for users but at least EPG data would be correct in the guide.

 

Hopefully other members chime in on this as well. Simply allowing us to manually map channels vs auto mapping based on arbitrary channel name would be a huge improvement. Couple that with allowing unmapping of a channel and we can fix any changes that are incorrectly mapped currently. Right now Emby is auto mapping on the channel number, not even the full channel number and name. 

 

So this channel: 

 

#EXTINF:-1 tvg-id="1009 ITV 4" tvg-name="1009 ITV 4" tvg-logo="https://themediaboxx...8/05/itv-4.png"group-title="Emby UK",1009 ITV 4

 

Is matching the 1009 only from the very end of the line. See attached image file. That doesn't make any sense. If you have 4 different guides you  could have channel 1009 in all 4 of them and it will match the first 1009 it sees in a guide. 

 

Removing auto mapping and allowing unmapping would resolve this in the short term. 

Attached Files



#7 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138075 posts
  • Local time: 04:30 AM

Posted 04 March 2019 - 02:15 PM

Right if it doesn't' find an exact match, then it looks for partial matches. In most cases this will work well for most users. When it doesn't that's when the channel management features will help. Thanks.



#8 Spaceboy OFFLINE  

Spaceboy

    Advanced Member

  • Members
  • 3891 posts
  • Local time: 09:30 AM

Posted 04 March 2019 - 02:18 PM

Right if it doesn't' find an exact match, then it looks for partial matches. In most cases this will work well for most users. When it doesn't that's when the channel management features will help. Thanks.

sorry I disagree Luke. For hdhr and hauppauge people the channels match by number. For those people using iptv this is a disaster and has been since you made this change. It’s not as though you haven’t been told a hundred times in the past

#9 IkeTaylor11 OFFLINE  

IkeTaylor11

    Advanced Member

  • Members
  • 393 posts
  • Local time: 03:30 AM

Posted 04 March 2019 - 02:26 PM

Right if it doesn't' find an exact match, then it looks for partial matches. In most cases this will work well for most users. When it doesn't that's when the channel management features will help. Thanks.

That is flawed logic, why would you ever want it to match on a partial? That would mean 1009 could match on 1009 or 09 or 100 or 009 all depending on what the ID's are coming over from GN. Then you have to make sure you add your GN guides in a top down order to make sure nothing is matched before you want it too. And you also have to make sure the same channel number isn't being pulled in from GN in multiple guides, which isn't possible because there is no filtering option for guide data being pulled from GN. But, even if that was that case and you gave us the ability to UNMAP channels and AUTO CHANNEL MAPPING wasn't enabled it would give us a resolution in the short term until "channel management" is a reality. This would "solve" the issue in the short term to satisfy paying customers even though it would mean work for us. 


Edited by IkeTaylor11, 04 March 2019 - 02:37 PM.

  • CBers likes this

#10 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138075 posts
  • Local time: 04:30 AM

Posted 04 March 2019 - 04:37 PM

Thanks for the feedback.



#11 IkeTaylor11 OFFLINE  

IkeTaylor11

    Advanced Member

  • Members
  • 393 posts
  • Local time: 03:30 AM

Posted 04 March 2019 - 04:51 PM

Thanks for the feedback.

 

What does that even mean? Which feedback are you referring to? Do you plan to act on any of our feedback? Or is this your way of ignoring the issue in hopes it just goes away? 



#12 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138075 posts
  • Local time: 04:30 AM

Posted 04 March 2019 - 06:16 PM

We are looking into improving our channel mapping and management. Thanks !



#13 cayars OFFLINE  

cayars

    Advanced Member

  • Alpha Testers
  • 2945 posts
  • Local time: 04:30 AM

Posted 13 March 2019 - 01:58 PM

sorry I disagree Luke. For hdhr and hauppauge people the channels match by number. For those people using iptv this is a disaster and has been since you made this change. It’s not as though you haven’t been told a hundred times in the past

I have to agree.  Emby handles OTA and Cable pretty well overall but still has issues.  Those issues however really standout when you try to add M3U files.

 

On a system I'm working on right now I've had to add 6 EPG sources and still need additional for full coverage.  However, with each new addition of an EPG provider things get more and more complicated and there is NO WAY to know what "automatic" channel mapping will take place.  Ordering of the EPG providers DOES MATTER since it will try to map channels based on channel number (should not do this ever, IMHO).

 

xTeVe is helpful but requires a ton of work especially since you need to manually change channel numbers from the 1000 starting number in order to avoid this auto channel mapping thing Emby does which is ugly. You need to go in and reorder ever channel to 2000+ (some high number) in order to avoid an auto-match of the EPGs based on channel number you'll likely find in the Emby EPG sources.  Needless to say this is a crazy exercise in frustration that really should not be needed even if it helps by working around the Emby auto match.

 

Even setting up xTeVe with 3rd party EPG sources (at additional cost) you have to edit nearly every channel number to bump it up past the highest channel you'll likely cover in your cable lineup or face mis-matches.  That can take days to do.

 

Even after getting this to work you still miss nice features that could be there.  For example if using multiple providers there is an overlap of channels.  This will likely be especially true if you also have OTA or cable as well.  What is badly needed is the ability to group channels regardless of source with an EPG match to the group and then being able to set a priority for each tuner/device/source such as always use (OTA, Cable, M3U-1, M3U-2) in that order if a tuner is busy or not responding.  This way if M3U-1 is down it rolls over and uses M3U-2 for the same channels.

 

Next PVR gets most of this correct with it's grouping, multi-rec and rollover tuner support, but unfortunately only allows 1 M3U file import so you can't add multiple M3U sources and control # of streams.  Since xTeVe isn't a true HDHomeRun emulator NPVR won't pick it up as a HDHomeRun and everything needs adding as the 1 M3U source (hence no backup of M3U sources).

 

Unfortunately no matter how you try to set this type of thing up in Emby it's going to be a hard proposition due to the way things work and it doesn't need to be this way.


Edited by cayars, 19 March 2019 - 03:07 PM.

  • IkeTaylor11 likes this

#14 cayars OFFLINE  

cayars

    Advanced Member

  • Alpha Testers
  • 2945 posts
  • Local time: 04:30 AM

Posted 13 March 2019 - 07:44 PM

All man, ready to throw in the towel.

After getting nearly complete with mapping, noticed a few Pac 12 channels in xTeVe weren't showing up and were being filtered out as they weren't marked as sports but as US regionals.  Changed the xTeVe filter and everything defaulted back to channel 1000 again.

 

Can't believe it doesn't internally match the channel number to the stream so any adjustment or difference in the M3U file doesn't change your channel numbers.

 

It shouldn't be this hard.  :)


Edited by cayars, 14 March 2019 - 07:16 AM.

  • CBers likes this

#15 CBers OFFLINE  

CBers

    Advanced Member

  • Moderators
  • 15014 posts
  • Local time: 09:30 AM
  • LocationKent, England.

Posted 14 March 2019 - 05:54 AM

All man, ready to throw in the towel.


:)

It's not the easiest thing to work with.

I spent quite a few hours to get my IPTV Channels into a state where they worked in Emby.

I was going to suggest the XTeVe discord, but I see you've already posted in there.

#16 cayars OFFLINE  

cayars

    Advanced Member

  • Alpha Testers
  • 2945 posts
  • Local time: 04:30 AM

Posted 14 March 2019 - 07:15 AM

Yea, I was fine using xTeVe right up until I changed a filter and noticed it wiped out my channel numbers.

Then I played and noticed if I add a channel in the M3U it "bumps" all my channels as well.

 

Then the fact that all channels start at 1000 causes me issues since the EPG data goes to around 1800 so there are lots of overlap channels.

 

I've got a bunch of hours invested as well and think I no all the things NOT TO DO now.  :)

 

 

Has anyone around the Emby forums written anything to import a M3U files into a database of some kind (SQL, MS Access, MySQL, etc)?

I'm thinking something like this might be the best approach as all manipulation can be done in the DB and then exported back as a new M3U file.

 

Could assign any "proper" tags this way including a non changing channel number if needed.



#17 IkeTaylor11 OFFLINE  

IkeTaylor11

    Advanced Member

  • Members
  • 393 posts
  • Local time: 03:30 AM

Posted 14 March 2019 - 07:39 AM

Yea, I was fine using xTeVe right up until I changed a filter and noticed it wiped out my channel numbers.
Then I played and noticed if I add a channel in the M3U it "bumps" all my channels as well.

Then the fact that all channels start at 1000 causes me issues since the EPG data goes to around 1800 so there are lots of overlap channels.

I've got a bunch of hours invested as well and think I no all the things NOT TO DO now. :)


Has anyone around the Emby forums written anything to import a M3U files into a database of some kind (SQL, MS Access, MySQL, etc)?
I'm thinking something like this might be the best approach as all manipulation can be done in the DB and then exported back as a new M3U file.

Could assign any "proper" tags this way including a non changing channel number if needed.


I use this software from a member here.

http://iptv-tools.com/

Can’t get my epg file to load though. But for m3u editing it works well.

There’s always xtream-editor.com as well.

But both of those don’t solve the real issue of using Emby Guide Data. The auto assign and not knowing what epg channel id to use for matching is still the problem.

If they would just take away auto assign and allow unmapping of channels it would solve 99% of the problem. No idea why it’s so difficult to get that across to them and the change to be made.


Sent from my iPhone using Tapatalk

#18 maegibbons OFFLINE  

maegibbons

    Advanced Member

  • Members
  • 2337 posts
  • Local time: 09:30 AM
  • LocationLutterworth, England, UK

Posted 14 March 2019 - 10:31 AM

Has anyone around the Emby forums written anything to import a M3U files into a database of some kind (SQL, MS Access, MySQL, etc)?

I'm thinking something like this might be the best approach as all manipulation can be done in the DB and then exported back as a new M3U file.

 

Carlo

 

I wrote a Access DB m3u writer last year to manipulate an m3u from our iptv provider here in the UK.  The issue was it was a Manual import.

 

This was used by a few users here in the UK

 

Last Month I wrote an m3u Parser to pull in and split up all the data from an incoming m3u file.

 

I am just in the process of writing the middleware to manipulate it however not too much time to spend on it.

 

Should be ready in a few to several weeks.

 

Krs

 

Mark


  • CBers likes this

#19 IkeTaylor11 OFFLINE  

IkeTaylor11

    Advanced Member

  • Members
  • 393 posts
  • Local time: 03:30 AM

Posted 14 March 2019 - 10:35 AM

Carlo

I wrote a Access DB m3u writer last year to manipulate an m3u from our iptv provider here in the UK. The issue was it was a Manual import.

This was used by a few users here in the UK

Last Month I wrote an m3u Parser to pull in and split up all the data from an incoming m3u file.

I am just in the process of writing the middleware to manipulate it however not too much time to spend on it.

Should be ready in a few to several weeks.

Krs

Mark


Very interested in this. Please keep us posted. Thank you.


Sent from my iPhone using Tapatalk

#20 cayars OFFLINE  

cayars

    Advanced Member

  • Alpha Testers
  • 2945 posts
  • Local time: 04:30 AM

Posted 14 March 2019 - 11:14 AM

http://iptv-tools.com/

Very similar to xTeVe and has the same or more issues.  It's basically a 1 time edit but doesn't reprocess M3U files again and again keeping your settings, channel numbers or allow you to merge multiple providers.

Carlo

 

I wrote a Access DB m3u writer last year to manipulate an m3u from our iptv provider here in the UK.  The issue was it was a Manual import.

 

This was used by a few users here in the UK

 

Last Month I wrote an m3u Parser to pull in and split up all the data from an incoming m3u file.

 

I am just in the process of writing the middleware to manipulate it however not too much time to spend on it.

 

Should be ready in a few to several weeks.

 

Krs

 

Mark

Yep, this is what I'm talking about.

 

That to me it the key.  You can load lists as often as you want into a DB and have both "input/mp3" columns plug have columns for your own modified data that you want to use when you write each record back out.

 

Flag records as not to export

Change Titles, Channel Number, Tags, etc

 

http://iptv-tools.com/

Very similar to xTeVe and has the same or more issues.  It's basically a 1 time edit but doesn't reprocess M3U files again and again keeping your settings, channel numbers or allow you to merge multiple providers.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users