Subtitle ANSI marking "?" on accented characters

[djhifi 56]

Post-disclaimer:

 

For the sake of clarity I'll state, from time to time, the current status of this bugfix, and will, therefore, mark this thread as "solved" once it's addressed.

 

Date of request/spotting : 25/September/2016

 

Current status (08/November/2016) :  UNFIXED

 

 

---------------------------------

 

BUG description:

 

Ive just realized this is a known linux server side bug. Any plans on solving it soon?

 

When using external ANSI subs and transcoding on a SYNOLOGY (edit: ANY Linux server machine), the emby server app is not displaying the accented characters correctly. Instead is marking them with "?"

 

Its temporarily solved by converting the subs to UTF-8, but:

 

This workaround doesnt work for me and others. Not only because i have a really big collection (5k movies 500 tv shows) but as well as the fact that it would render direct play on KODI, with problems given the fact UTF-8 is not used for portuguese language in external subtitles and roughly, 50 other alphabets. When we convert the subtitles to UTF-8, it f*cks them up (i.e. the whole text) and I mean LITERALLY

 

You can see it thoroughly explained in here:

 

https://emby.media/community/index.php?/topic/39130-playback-subtitles-weird-display/

 

EDIT:

 

To quickly wrap it up,

 

When we stream videos and use EXTERNAL subtitles encoded in ANSI (which is the standard encode for external subs on France, Italy, Portugal, Spain, etc), ALL users, will get "?" instead of "á" "ão" "ó", etc. Because the SYNOLOGY EMBY SERVER APP has this NOT FIXED. It's solved by converting the subtitles to UTF-8 (mainly used on English speaking countries - no accented chars- ), but it then renders direct play (samba direct stream) unplayable as it messes with the whole body of the text. Furthermore, it's the definition of pain in the ass for people (like me) that have over 5000 movies and 540 tv shows to re-converted all subs. Unpraticable.

Edited November 8, 2016 by djhifi

[Luke 39294]

Hi, it's being looked into, thanks.

[djhifi 56]

Hi, it's being looked into, thanks.

 

@@Luke

 

Thanks for the attention given. Is this already sorted out on the new release of the server app on SYNOLOGY? I ask this because either it's not or I am not turning on a specific option that may be implemented??

Edited October 8, 2016 by djhifi

[Luke 39294]

Probably not yet.

[djhifi 56]

@@Luke

 

Are you dealing with this implementation/fix yourself or?

 

Just to clear things out. It really is something I needed...I do a lot of outside streaming when I'm not home and the subs are junk atm

Edited October 18, 2016 by djhifi

[djhifi 56]

@@Luke

 

I've also found out another thread which reports this:

 

 

https://emby.media/community/index.php?/topic/7719-external-subtitles-ansi/page-3&do=findComment&comment=366722

 

A quick re-wrap of the problem for newcomers:

 

When we stream videos and use external subtitles encoded in ANSI (which is the standard encode for external subs on France, Italy, Portugal, Spain, etc), ALL users, will get "?" instead of "á" "ão" "ó", etc. Because the SYNOLOGY EMBY SERVER APP has this NOT FIXED. It's solved by converting the subtitles to UTF-8 (mainly used on English speaking countries - no accented chars- ), but it then renders direct play (samba direct stream) unplayable. Furthermore, it's the definition of pain in the ass for people (like me) that have over 5000 movies and 540 tv shows to re-converted all subs. Unpraticable.

Edited October 25, 2016 by djhifi

[Abobader 3290]

Good day,

 

Yes, it been reporting before, as for arabic subtitles, still issue tho.

 

My best

[solabc16 379]

Hello @@djhifi

 

Can you post an example SRT file from your system here or PM it to me.

 

Best

- James

[djhifi 56]

Hello @@djhifi

 

Can you post an example SRT file from your system here or PM it to me.

 

Best

- James

 

 

@@solabc16 No need to post an example. Just create an .srt file (or modify an existing one) and add it: a) accented characters & save it in ANSI format.

 

example for accented characters:

 

ã

ão

é

á

ó

Edited November 1, 2016 by djhifi

[solabc16 379]

Hello @@djhifi

 

I'm aware I can do this, but I'd rather work with a file that you are actively trying to get working on your system, so we have a common reference and baseline to work from.

 

- James

[djhifi 56]

Hello @@djhifi

 

I'm aware I can do this, but I'd rather work with a file that you are actively trying to get working on your system, so we have a common reference and baseline to work from.

 

- James

 

@@solabc16, the problem happens ON ALL .srt files in my system as ALL are encoded in ANSI format and use accented characters. But if you still want here it is:

 

http://www.filedropper.com/independencedayressurgence2016

 

INDEPENDENCE DAY RESSURGENCE 2016 (subtitles are synced with the REMUX VERSION of the release but IMO can work on scene encodes i think).

 

PS: It's strange that Luke said "it's being looked into" 2 months ago and, since then, he stopped answering my forum quotes and even my PM's...

Edited November 1, 2016 by djhifi

[solabc16 379]

Hello @@djhifi

 

Thanks, I want to get to the bottom of this, so we're going to have to work together and likely cover some 'old' ground.

 

The best way is if we work with a specific title and file, so we know we're making the right conclusions.

 

In terms of setup, what do you have on each end?

 

i.e what server platform(s) and what clients, including versions.

 

- James

[djhifi 56]

Hello @@djhifi

 

Thanks, I want to get to the bottom of this, so we're going to have to work together and likely cover some 'old' ground.

 

The best way is if we work with a specific title and file, so we know we're making the right conclusions.

 

In terms of setup, what do you have on each end?

 

i.e what server platform(s) and what clients, including versions.

 

- James

 

@@solabc16 I have a SYNOLOGY 1813+ server side. And I use for streaming the external access of that same server side EMBY or the EMBY THEATER app.

 

On my home I have 2 kodi machines 16.1 with LibreELEC installed in both. Altough there is no problem here. Because I made path substitution and I use samba stream (direct streaming so, no transcode needed). Subs are displayed fine.

 

Problem comes in outside streaming (or even streaming in-home) as long as I stream...subtitles are presented with problems. And this is because the issue is only server side (emby for linux side). Samba + KODI is fine.

 

But KODI also displays problems if I set "transcode" in EMBY FOR KODI addon. But I dont do that...as I mentioned.

 

The bug is server side. As I always said.

 

And only happens in Linux server versions btw...dunno why

[barat 27]

It happens on Linux and Synology (DS916+) ... my Polish subtitles are messed up as well - no mater if I watch on KODI or on Windows 10 Firefox ... funny thing is that not all subtitles, some embedded into mkv works OK...

[Angelblue05 4131]

Do your subs contain the language tag in the filename? I thought I put in a temp fix for direct play from HTTP in emby for kodi. Is this not the case? Or are you transcoding?

 

 

Sent from my iPhone using Tapatalk

Edited November 7, 2016 by Angelblue05

[djhifi 56]

It happens on Linux and Synology (DS916+) ... my Polish subtitles are messed up as well - no mater if I watch on KODI or on Windows 10 Firefox ... funny thing is that not all subtitles, some embedded into mkv works OK...

 

 

As I said, encoded subtitles INSIDE .mkv's are mainly in UTF-8. Which displays fine when streaming from the server. The bug is with external subtitles in ANSI + server streaming (transcoding).

 

The body of your external UTF-8 subtitles is f*cked up in Windows (I bet) because you, like me, live in a country where the alphabet uses accented characters, and for encoding purposes it must be set to ANSI. Try setting to ANSI and puff, problem of f*cked up subs goes away but then, you have ??? in the place of  á é í ão ó à bé mãe joão, etc, etc... as long as you stream from the server (make the transcode in the machine where EMBY is installed).

 

If you don't transcode from the server (meaning, you DIRECTLY STREAM to other client, nothing happens). As I said 1232143 times, this is a bug in EMBY Server for Linux (meaning ALL systems with Linux, such as FreeBSD or yours and my SYNOLOGIES will have the bug). Which, in my case, keeps me from viewing movies and tv shows when I'm not home.

 

This is not happening on Windows servers (god knows why).

 

@@Luke said "it's being looked into". But so far...nothing and no more replies were give (even tough, from time to time, I @ him)

 

Altough this is 2016 and ANSI is read everywhere...we still need to wait it seems.

 

In my opinion, this is the BEST software out there. But (and I already made those feature requests), lacks very much in BASIC subtitle support such as: ANSI support (and other encodes), subtitle positioning, coloring, font size, etc.

 

None has made it's way into the program yet...which is a pity.

 

This (the bug, mainly) is really a pain for a huge number of users at this moment (I wonder!) and, keeps me away of Lifetime Donation at this moment, as well.

 

I will regularly update the 1st post as to whether or not this has been fixed.

 

For the time being:

 

Welcome aboard! 

Edited November 8, 2016 by djhifi

[solabc16 379]

Hello All

 

If the assertion that this does not happen under any circumstance on Windows holds true, then the cause will likely be due to the different runtimes on which Emby Server is running.

 

On Windows, Emby Server runs on the Microsoft .NET runtime, whilst on all *NIX based systems it uses the Mono cross platform, open source .NET framework.

 

That being the case, it could be that a relied on behaviour differs in its default implementation between the environments or that the behaviour is simply undefined.

 

Either way, that's what we're working on getting to the bottom of and given a finite set of time and resource that will take as long as it takes; but it is being looked at.

 

Regarding the statement that converting the files is not viable, that only holds true if you are trying to do it manually. If that is a vialbe workaround that people would be happy to accept as an interim, we can write a utility to include with the package that will do this automatically. It certainly won't take long for the h/w platform you're running on to make the conversion and is something we could put together quite quickly. Do we know that doing that doesn't cause issues elsewhere? (i.e. do all external players in use support UTF-8 properly?)

 

Best

- James

 

@@djhifi, I appreciate that this might be a significantly frustrating issue for you and that it's been that way for some time. I want to get to the bottom of this and find a solution as well, as it's a potential blocker to adoption for a significant number of users. However, can I please ask that we dial it back a little and drop the agressive posts, subsequent edits and block capitals. None of this does anything to garner the support of the development community and get them on your side.

[solabc16 379]

Hello

 

So here's the output from some controlled tests across different versions, browsers and platforms (as a control):-

 

Emby Server, Stable 3.0.8500, Synology, Firefox

 

 

Emby Server, Stable 3.0.8500, XPEnology, Firefox

 

 

Emby Server, Stable 3.0.8500, Synology, Chrome

 

 

Emby Server, Stable 3.0.8500, XPEnology, Chrome

 

 

Emby Server, Beta 3.1.211, XPEnology, Firefox

 

>> External subtitles do not work.

 

Emby Server, Beta 3.1.211, XPEnology, Chrome

 

 

@@Luke, I haven't tracked down anything in the release notes, is this a side effect or intentional?

 

Best

- James

Edited November 8, 2016 by solabc16

[Luke 39294]

Is what intentional?

[solabc16 379]

Is what intentional?

 

Hello

 

That this appears to be (partially) fixed in 3.1.211-beta. There's no mention of it that I can find in the release notes, but the behaviour is different as documented above.

 

- James

[Luke 39294]

No, not in the beta but the last stable had a change that may have helped. So it would have already been in beta recently.

[solabc16 379]

No, not in the beta but the last stable had a change that may have helped. So it would have already been in beta recently.

 

Ok, well that's not what I'm seeing in the controlled test cases. If you follow the annotations to each screen grab above, you'll see that it is not working on stable but works correctly on beta (on Chrome at least, Firefox is broken completely).

 

- James

[solabc16 379]

Here's an equivalent test, using Emby Theater as the client. The same behaviour is observed:-

 

Emby Server, Stable 3.0.8500, XPEnology, Emby Theater (2.5.28)

 

 

Emby Server, Beta 3.1.211, XPEnology, Emby Theater (2.5.28)

 

 

There appears to be a problem with font size and positioning, but that's not the primary issue I'm looking at here.

 

- James

[Mock 7]

I live in Denmark and have the same issue, it a bit of a show stopper for me on keep using Emby.

 

My config is Synology DS1515+ > Nvidia Shield SPMC and Kodi for Windows. my current workaround is that when playing the movie there is two sets of Subtitles where one of them has the ? and the other look fine. im not aware if its the External/downloaded subtitle that works or what going on. but it looks like the same issue that other have. Hope some one comes up with a fix on this issue

[solabc16 379]

Hello @@Mock

 

Agree, it's something we have to get resolved as it is a potential blocker for a number of users.

 

If you look through the earlier posts, you'll see the screenshots from a recent set of test cases.

 

You'll notice that the on the latest beta (3.1.211 at the time) the problematic characters are displaying correctly, in a side-by-side test.

 

So there is progress.

 

You are welcome to try out the beta and see how it works for you, but there are caveats - please read this article on the Wiki: https://github.com/MediaBrowser/Wiki/wiki/Synology-:-Accessing-Beta-and-Development-Releases

 

- James