emby downloads hundreds of the same subtitles for movies

[drevokocur 0]

I have a strange problem. For some movies (say 50% of all my movies), Emby continuously downloads subtitles, even if they are already downloaded. So I'm ending up with a movie folder with hundreds of the same subtitles, just with an increasing suffix number. Any idea why? 

Edited October 26, 2020 by drevokocur

[Luke 42078]

Hi there, can you please go over an example and attach the information requested in how to report a problem? Thanks !

[billzkid 1]

Hi, 

I will try to further clarify the problem because I have the same issue. I just posted a snippet from a log i think it is relevant. I know you prefer whole logs so if this will not be enough i will provide whole logs.

I am using Emby 4.5.2.0 as can be seen bellow in the App Data section from server log.

The symptom of the error is, as drevokocur said, hundreds of subtitles for certain movies in the media folder. As you can see in the log excerpt below it seems subtitle manager tried to download subtitles twiceand saved 365-th and 366-th version of the subtitles for this movie then moved forward to next subtitles. I have upload screenshot of the folder with multiple subtitles as well.

I have also attached screenshot of subtitles settings of the library. Slovenian and English languages are selected (can't be seen on screenshot).

Is this ok (do you have enough info)?

 

2020-10-30 07:36:30.297 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-10-30 07:36:30.356 Info HttpClient: GET https://www.podnapisi.net/subtitles/search/old?sXML=1&sL=sl&sK=Her Smell&sY=2019
2020-10-30 07:36:30.937 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-10-30 07:36:30.994 Info HttpClient: GET https://www.podnapisi.net/subtitles/search/old?sXML=1&sL=sl&sK=Hollywood Homicide&sY=2003
2020-10-30 07:36:31.563 Info HttpClient: GET https://www.podnapisi.net/sl/subtitles/sl-hollywood-homicide-2003/5gQ/download
2020-10-30 07:36:31.968 Info SubtitleManager: Saving subtitles to F:\Movies\MoviesToWatchLater\Hollywood.Homicide.2003.720p.BluRay.x264-x0r\Hollywood.Homicide.2003.720p.BluRay.x264-x0r.365.sl.sub
2020-10-30 07:36:31.983 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffprobe.exe -i file:"F:\Movies\MoviesToWatchLater\Hollywood.Homicide.2003.720p.BluRay.x264-x0r\Hollywood.Homicide.2003.720p.BluRay.x264-x0r.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2020-10-30 07:36:32.034 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0
2020-10-30 07:36:32.295 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-10-30 07:36:32.359 Info HttpClient: GET https://www.podnapisi.net/subtitles/search/old?sXML=1&sL=sl&sK=Hollywood Homicide&sY=2003
2020-10-30 07:36:33.003 Info HttpClient: GET https://www.podnapisi.net/sl/subtitles/sl-hollywood-homicide-2003/5gQ/download
2020-10-30 07:36:33.328 Info SubtitleManager: Saving subtitles to F:\Movies\MoviesToWatchLater\Hollywood.Homicide.2003.720p.BluRay.x264-x0r\Hollywood.Homicide.2003.720p.BluRay.x264-x0r.366.sl.sub
2020-10-30 07:36:33.792 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-10-30 07:36:33.849 Info HttpClient: GET https://www.podnapisi.net/subtitles/search/old?sXML=1&sL=sl&sK=Holy Hell&sY=2016
2020-10-30 07:36:34.322 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc

 

App Data

2020-10-30 00:00:00.020 Info App: Application version: 4.5.2.0
2020-10-30 00:00:00.020 Info App: Emby
Command line: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
Operating system: Microsoft Windows 10.0.19041
Framework: .NET Core 3.1.8
OS/Process: x64/x64
Runtime: C:/Users/billzkid/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
Processor count: 8
Data path: C:\Users\billzkid\AppData\Roaming\Emby-Server\programdata
Application path: C:\Users\billzkid\AppData\Roaming\Emby-Server\system
2020-10-30 00:00:00.020 Info App: Plugins:
Bluray Folder Support 1.0.0.0
Cinema Intros 1.0.35.0
Dlna 1.0.39.0
Dvd Folder Support 1.0.0.0
Fanart.tv 1.0.10.0
MovieDb 1.3.8.0
MusicBrainz 1.0.13.0
Nfo Metadata 1.0.34.0
OMDb 1.0.11.0
Open Subtitles 1.0.27.0
Podnapisi 1.0.6.0
Port Mapper 1.0.4.0
Studio Images 1.0.3.0
TheAudioDb 1.0.11.0
TheTVDB 1.0.35.0
Trailers 1.2.9.0
Webhooks 1.0.13.0
2020-10-30 00:03:17.619 Info SoftwareCodecProvider: h264, libx264, libx264 Software Encoder, V-E-libx264
2020-10-30 00:03:17.619 Info SoftwareCodecProvider: hevc, libx265, libx265 Software Encoder, V-E-libx265
2020-10-30 00:03:17.619 Info SoftwareCodecProvider: mpeg4, mpeg4, MPEG-4 part 2 Software Encoder, V-E-mpeg4
2020-10-30 00:03:17.619 Info SoftwareCodecProvider: msmpeg4v3, msmpeg4, MPEG-4 part 2 (MS Variant 3) Software Encoder, V-E-msmpeg4
2020-10-30 00:03:17.619 Info SoftwareCodecProvider: vp8, libvpx, libvpx VP8, V-E-libvpx
2020-10-30 00:03:17.621 Info SoftwareCodecProvider: h264, libx264, libx264 Software Encoder, V-E-libx264
2020-10-30 00:03:17.621 Info SoftwareCodecProvider: hevc, libx265, libx265 Software Encoder, V-E-libx265
2020-10-30 00:03:17.621 Info SoftwareCodecProvider: mpeg4, mpeg4, MPEG-4 part 2 Software Encoder, V-E-mpeg4
2020-10-30 00:03:17.621 Info SoftwareCodecProvider: msmpeg4v3, msmpeg4, MPEG-4 part 2 (MS Variant 3) Software Encoder, V-E-msmpeg4
2020-10-30 00:03:17.621 Info SoftwareCodecProvider: vp8, libvpx, libvpx VP8, V-E-libvpx
2020-10-30 00:03:17.622 Info Dx11VaCodecProvider: Start GetCodecList()
2020-10-30 00:03:19.214 Info Dx11VaCodecProvider: End GetCodecList()
2020-10-30 00:03:19.214 Info QuickSyncCodecProvider: ProcessRun 'ffdetect_qsvencdec' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 qsvencdec -print_format json -disable_dx11
2020-10-30 00:03:19.920 Info QuickSyncCodecProvider: ProcessRun 'ffdetect_qsvencdec' Process exited with code 0
2020-10-30 00:03:19.981 Info QuickSyncDx11CodecProvider: ProcessRun 'ffdetect_qsvencdec' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 qsvencdec -print_format json
2020-10-30 00:03:20.137 Info QuickSyncDx11CodecProvider: ProcessRun 'ffdetect_qsvencdec' Process exited with code 0
2020-10-30 00:03:20.197 Info NvidiaCodecProvider: ProcessRun 'ffdetect_nvencdec' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 nvencdec -print_format json -disable_dx11
2020-10-30 00:03:20.232 Info NvidiaCodecProvider: ProcessRun 'ffdetect_nvencdec' Process exited with code 1
2020-10-30 00:03:20.291 Info AmfCodecProvider: ProcessRun 'ffdetect_amfenc' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 amfenc -print_format json
2020-10-30 00:03:20.515 Info AmfCodecProvider: ProcessRun 'ffdetect_amfenc' Process exited with code 0
2020-10-30 00:03:20.574 Info AmfCodecProvider: Adapter #0: 'AMD Radeon(TM) RX Vega 11 Graphics' Id:5597 (Driver: , Vendor: 4098) D3D11
2020-10-30 00:03:20.575 Info AmfCodecProvider: Adapter #1: 'AMD Radeon(TM) RX Vega 11 Graphics [1]' Id:5597 (Driver: , Vendor: 4098) D3D11
2020-10-30 00:03:20.623 Info SoftwareCodecProvider: h264, libx264, libx264 Software Encoder, V-E-libx264
2020-10-30 00:03:20.623 Info SoftwareCodecProvider: hevc, libx265, libx265 Software Encoder, V-E-libx265
2020-10-30 00:03:20.623 Info SoftwareCodecProvider: mpeg4, mpeg4, MPEG-4 part 2 Software Encoder, V-E-mpeg4
2020-10-30 00:03:20.623 Info SoftwareCodecProvider: msmpeg4v3, msmpeg4, MPEG-4 part 2 (MS Variant 3) Software Encoder, V-E-msmpeg4
2020-10-30 00:03:20.623 Info SoftwareCodecProvider: vp8, libvpx, libvpx VP8, V-E-libvpx
2020-10-30 00:03:20.625 Info SoftwareCodecProvider: h264, libx264, libx264 Software Encoder, V-E-libx264
2020-10-30 00:03:20.625 Info SoftwareCodecProvider: hevc, libx265, libx265 Software Encoder, V-E-libx265
2020-10-30 00:03:20.625 Info SoftwareCodecProvider: mpeg4, mpeg4, MPEG-4 part 2 Software Encoder, V-E-mpeg4
2020-10-30 00:03:20.625 Info SoftwareCodecProvider: msmpeg4v3, msmpeg4, MPEG-4 part 2 (MS Variant 3) Software Encoder, V-E-msmpeg4
2020-10-30 00:03:20.625 Info SoftwareCodecProvider: vp8, libvpx, libvpx VP8, V-E-libvpx
2020-10-30 00:03:20.626 Info Dx11VaCodecProvider: Start GetCodecList()
2020-10-30 00:03:22.234 Info Dx11VaCodecProvider: End GetCodecList()
2020-10-30 00:03:22.235 Info QuickSyncCodecProvider: ProcessRun 'ffdetect_qsvencdec' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 qsvencdec -print_format json -disable_dx11
2020-10-30 00:03:22.886 Info QuickSyncCodecProvider: ProcessRun 'ffdetect_qsvencdec' Process exited with code 0
2020-10-30 00:03:22.945 Info QuickSyncDx11CodecProvider: ProcessRun 'ffdetect_qsvencdec' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 qsvencdec -print_format json
2020-10-30 00:03:23.102 Info QuickSyncDx11CodecProvider: ProcessRun 'ffdetect_qsvencdec' Process exited with code 0
2020-10-30 00:03:23.166 Info NvidiaCodecProvider: ProcessRun 'ffdetect_nvencdec' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 nvencdec -print_format json -disable_dx11
2020-10-30 00:03:23.201 Info NvidiaCodecProvider: ProcessRun 'ffdetect_nvencdec' Process exited with code 1
2020-10-30 00:03:23.259 Info AmfCodecProvider: ProcessRun 'ffdetect_amfenc' Execute: C:\Users\billzkid\AppData\Roaming\Emby-Server\system\ffdetect.exe -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 amfenc -print_format json
2020-10-30 00:03:23.481 Info AmfCodecProvider: ProcessRun 'ffdetect_amfenc' Process exited with code 0
2020-10-30 00:03:23.540 Info AmfCodecProvider: Adapter #0: 'AMD Radeon(TM) RX Vega 11 Graphics' Id:5597 (Driver: , Vendor: 4098) D3D11
2020-10-30 00:03:23.541 Info AmfCodecProvider: Adapter #1: 'AMD Radeon(TM) RX Vega 11 Graphics [1]' Id:5597 (Driver: , Vendor: 4098) D3D11

 

Symptom in the folder

 

Library settings:

Edited October 30, 2020 by billzkid

[Luke 42078]

Hi, this is resolved for the next release of the server. Thanks.

[drevokocur 0]

thanks guys!

[gberg 6]

I also have this issue, and one thing I've been thinking about is movies located on storage whe the Emby server doesn't have whrite access, then the subtitle files gets stored on the server instead, but in which folder are the subtitle files stored?

My Emby server is running Ubuntu 18.04.

[Luke 42078]

13 hours ago, gberg said:

I also have this issue, and one thing I've been thinking about is movies located on storage whe the Emby server doesn't have whrite access, then the subtitle files gets stored on the server instead, but in which folder are the subtitle files stored?

My Emby server is running Ubuntu 18.04.

In the media folder if you enabled that, otherwise the server's internal metadata folder.

[gberg 6]

6 hours ago, Luke said:

In the media folder if you enabled that, otherwise the server's internal metadata folder.

Yes, subtitles are set to be stored in the media folder, but for folders when Emby server doesn't have write access the subtitles must be stored somewhere on the server, but I don't know the location of the files, in what folder on the server are they stored?

[Happy2Play 9780]

14 hours ago, gberg said:

Yes, subtitles are set to be stored in the media folder, but for folders when Emby server doesn't have write access the subtitles must be stored somewhere on the server, but I don't know the location of the files, in what folder on the server are they stored?

They should be buried in /metadata/library/xx/xxxxxxxxxxxxxxxxxxxxxxx folders.  You should see the location in your log as it should tell you can't write with media and falling to said location.

[gberg 6]

9 hours ago, Happy2Play said:

They should be buried in /metadata/library/xx/xxxxxxxxxxxxxxxxxxxxxxx folders.  You should see the location in your log as it should tell you can't write with media and falling to said location.

Ok, thanks!

[drevokocur 0]

On 31/10/2020 at 20:06, Luke said:

Hi, this is resolved for the next release of the server. Thanks.

@Lukehas this been resolved? I'm on 4.5.2.0. and the problem still occurs. thanks

[gberg 6]

34 minutes ago, drevokocur said:

@Lukehas this been resolved? I'm on 4.5.2.0. and the problem still occurs. thanks

No, it's not solved yet, but I've bee told in theis thread it's going to be solved in the next release:

 

But on the other hand I  was told it should have been solved in the next release earlier, when it in fact wasn't solved.

So we will just have to hope for it to be solved in the upcomming releas whenever it wil come... 

But this really needs to be solved, I have movies with several thousand of subtilte files, and I atached a image showing that.

[Luke 42078]

Repeated downloads in the .sub format should be resolved in 4.5.3. Thanks.

[gberg 6]

5 hours ago, Luke said:

Repeated downloads in the .sub format should be resolved in 4.5.3. Thanks.

Great! Any ideas when that will be available?

[Luke 42078]

Soon. Stay tuned to the blog for the announcement. Thanks.

[gberg 6]

Ah, I see version 4.5.3 is out now, I'll upgrade now and see how it goes.

[gberg 6]

I hate to say it, but it seems the issue is still there, at least when it comes to subtitles in .txt  and .srt format.

I manually started the Download Subtitles  Sheduled Task, and it only at about 5% yet but still it has downloadet subtiltles for some movies that already got a couple of hundreds of subtitle fllies, and that is after I upgraded to 4.5.3 and rebooted the server.

[Carlo 4561]

Hi, Is this happening in both movies and tv shows or just movies?

Do you have one movie per folder setup on your system?

Have you done a full library scan BEFORE running the subtitle download job?

Edited December 4, 2020 by cayars

[gberg 6]

5 hours ago, cayars said:

Hi, Is this happening in both movies and tv shows or just movies?

Do you have one movie per folder setup on your system?

Have you done a full library scan BEFORE running the subtitle download job?

I have only seen this on movies.

Yes, I have one movie per folder.

What do you mean by full librabry scan? My "Scan media library"  is scheduled to runs every three hours, and my "Download subtitle"  schedule runs  every day at midnigt. 

 

[Luke 42078]

There could still be an issue with .txt files, but .sub should be fine now.

[Carlo 4561]

37 minutes ago, gberg said:

What do you mean by full librabry scan? My "Scan media library"  is scheduled to runs every three hours, and my "Download subtitle"  schedule runs  every day at midnigt. 

From admin click on Library menu then Scan Library Files or just kick off the job that runs by schedule manually.

[gberg 6]

13 hours ago, Luke said:

There could still be an issue with .txt files, but .sub should be fine now.

Ok, will there be similar fix for .txt and .sub files too, or would it even be possible to fix the issue regardless of the file exension?

12 hours ago, cayars said:

From admin click on Library menu then Scan Library Files or just kick off the job that runs by schedule manually.

Yes, that I've done.

Edited December 5, 2020 by gberg

[Luke 42078]

Different problems. We don't support txt subtitle files, and their format is not really standardized which makes it difficult.

But we allow downloading them and then use ffmpeg to convert to set, which we do support. If that conversion fails, then that causes this.

Easy fix would be to never download txt subtitles, but then a lot of potential downloads would be lost.

So the conversion failures have to be looked at individually one at a time, with that hope that with enough fixes it will become something that is only seen rarely.

[gberg 6]

3 hours ago, Luke said:

Different problems. We don't support txt subtitle files, and their format is not really standardized which makes it difficult.

But we allow downloading them and then use ffmpeg to convert to set, which we do support. If that conversion fails, then that causes this.

Easy fix would be to never download txt subtitles, but then a lot of potential downloads would be lost.

So the conversion failures have to be looked at individually one at a time, with that hope that with enough fixes it will become something that is only seen rarely.

I have no idea how the subtitle download process works in Emby, but why does it matter what file extension the subtitle files have regarding the issue of downloading multiple copies for the same movie?

I mean if a movie already have a subtitle file of a certain language Emby shouldn't downoad any subtiitles of that particular language again, at least not automatically?

[Luke 42078]

Just now, gberg said:

I have no idea how the subtitle download process works in Emby, but why does it matter what file extension the subtitle files have regarding the issue of downloading multiple copies for the same movie?

I mean if a movie already have a subtitle file of a certain language Emby shouldn't downoad any subtiitles of that particular language again, at least not automatically?

We should only download subtitles that we support playing, otherwise they'd get downloaded and then never show up because they'd get ignored by the library scan.