Jump to content
somy

Possible for Emby not to download subtitles repeatedly?

Recommended Posts

somy

Hi,

 

I noticed that Emby keeps downloading subtitles for movies that already have subtitles (which were downloaded by Emby previously). 

I schedule the subtitle download everyday and noticed in many of the movie folders there are hundreds of them (I configure Emby to save the downloaded subtitle to media folder).

Shouldn't Emby not try to download subtitle for the movies that it already fetched subtitle before?

 

Thanks!

Share this post


Link to post
Share on other sites
Luke

Hi there, can you please go over an example? Thanks.

Share this post


Link to post
Share on other sites
somy

Hi there, can you please go over an example? Thanks.

Hi Luke, please see an example below, basically when I run download subtitle task, every time it downloads 4 identical subtitles and save them into the media folder. This folder has been cleaned up so only 16 duplicated subtitles (there were several hundreds when I noticed it). It happens with all other movies as well (whenever Emby can find subtitle from opensubtitles).

post-4441-0-34705700-1591517496_thumb.png

 

Below is the library settings for subtitles:

post-4441-0-68049800-1591517488_thumb.png

 

The log for this particular movie:

2020-06-07 09:59:54.350 Info SubtitleManager: Saving subtitles to /share/CACHEDEV3_DATA/Movies/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE.13.zh-cn.srt

2020-06-07 09:59:54.656 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-06-07 09:59:54.977 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-06-07 09:59:55.077 Info SubtitleManager: Saving subtitles to /share/CACHEDEV3_DATA/Movies/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE.14.zh-cn.srt
2020-06-07 09:59:55.153 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /share/CACHEDEV1_DATA/.qpkg/EmbyServer/bin/ffprobe -i file:"/share/CACHEDEV3_DATA/Movies/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2020-06-07 09:59:55.415 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0
2020-06-07 09:59:55.695 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-06-07 09:59:56.005 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-06-07 09:59:56.093 Info SubtitleManager: Saving subtitles to /share/CACHEDEV3_DATA/Movies/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE.15.zh-cn.srt
2020-06-07 09:59:56.401 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-06-07 09:59:56.709 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2020-06-07 09:59:56.841 Info SubtitleManager: Saving subtitles to /share/CACHEDEV3_DATA/Movies/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE/A.Shaun.the.Sheep.Movie.Farmageddon.2019.2160p.UHD.BluRay.X265-IAMABLE.16.zh-cn.srt

 

 

There are three issues I think:

  • The movie in this example already contains subtitles, and I select the option not to download subtitle when embedded subtitle exists. In my understanding this should prevent Emby try to download subtitle for this particular movie
  • Emby doesn't seem to remember it has already downloaded subtitles, and therefore keeps filling the disk with the same subtitles every time when the job runs
  • For this particular example Emby downloads the same subtitles 4 times (in the same language). Maybe a feature request: It would be nice for Emby to check the hash value of the downloaded subtitles and remove the duplicates before saving to disk

I checked the timestamp of the subtitles and it seems to indicate this problem started from April. Could be a bug introduced in 4.4.2 or 4.4.1.

Share this post


Link to post
Share on other sites
somy

Please let me know if you need more info, it's getting annoying to clean the files on daily basis :(

Thanks!

Share this post


Link to post
Share on other sites
trashken

Having the same problem too. Subtitles keep replicating like a virus.

Share this post


Link to post
Share on other sites
cam1313

Same here.  I have over 2,500 movies some with nearly 100 subtitles each...if anyone has any good ideas on ways to get rid of all the duplicate subtitles without going through each and every folder individually I'm open for suggestions.  TIA!

 

...also, obviously very interested in solving the root of this problem as well. 

Share this post


Link to post
Share on other sites
Luke
On 6/8/2020 at 4:46 AM, somy said:

Please let me know if you need more info, it's getting annoying to clean the files on daily basis :(

Thanks!

Please try this in Emby Server 4.4.3. Thanks.

Share this post


Link to post
Share on other sites
Luke
On 6/28/2020 at 10:51 PM, trashken said:

Having the same problem too. Subtitles keep replicating like a virus.

Hi there, can you please go over an example? Thanks !

Share this post


Link to post
Share on other sites
GPlayer
On 7/6/2020 at 3:33 AM, Luke said:

Hi there, can you please go over an example? Thanks !

I am also still having this problem. In certain files it will keep downloading a subtitle file every time it does a scan. On one file I hadn't noticed it had downloaded nearly 380 subtitle files.

Share this post


Link to post
Share on other sites
cayars

Can you provide a file name and your basic settings used to download subs.

I'd like to try and recreate this.

Thanks

Share this post


Link to post
Share on other sites
cam1313
6 hours ago, cayars said:

Can you provide a file name and your basic settings used to download subs.

I'd like to try and recreate this.

Thanks

For example, Emby has downloaded 199 versions of the same English subtitle for the Movie "Lucky Numbers (2000)."

 Under my movie library's subtitle downloader settings I have (listed in order of priority) Addic7ed, NapiSub, SubDB, Pdnapisi, and Open Subtitles selected, as well as the options to require a has match and save subtitles into media folders. 

Share this post


Link to post
Share on other sites
cayars
Posted (edited)

Manually remove all of them from that movie.  Then try removing Pdnapisi and do a library refresh on two.

I've seen this happen to a few people and removing that plugin seems to stop this from happening.

Edited by cayars

Share this post


Link to post
Share on other sites
cam1313

I’ll give that a shot. Do you have any recommendations for removing the multiple subtitles from the thousands of other movies with the same issue? If removing pdnapisi prevents the duplicate subtitles from redownloading that would solve the bigger issue, but in the event it doesn’t I’m also trying to come up with an effective way to clean up the multiple subtitles when this happens. Thanks in advance for any ideas. 

Share this post


Link to post
Share on other sites
cayars

You could probably script this or pull the info out of the database.

Let's see if we can find/fix the issue first on just a couple of test movies that you can manually remove the SRT from.

Share this post


Link to post
Share on other sites
ebr

Is this a confirmed issue in that plug-in?

Share this post


Link to post
Share on other sites
cayars

One or two other people have mentioned this from memory but I don't remember where I read it.

I'm trying to reproduce myself on my test setup.

Share this post


Link to post
Share on other sites
tmb4016
Posted (edited)

Hi,

I am not sure this applies to the original post situation but I have/had this issue as well but only on movies which are represented as media stubs and as such are named with .DVD.disc or .BLURAY.disc in the file name.

The screenshot of the original post looks like it actually is an mkv file but it does have the .BLURAY in the filename so it may be related.

I am not able to test anything at the moment but maybe useful info but from memory it did stop around end of April. I am on the Beta 4.5 now but cannot remember when this was updated.

 

 

Edited by tmb4016

Share this post


Link to post
Share on other sites
cayars

Thanks for that additional info.

Share this post


Link to post
Share on other sites
cam1313

None of my movie files have .DVD.disc or .BLURAY.disc in the filename, however, it does not appear removing the Pdnapisi plugin resolved the issue.  So far I've noticed that if I clear all but one of the subtitles it will download one additional duplicate, however, when I delete ALL the subtitles in the movie folder and refresh the library it will download just one.  I'm not yet convinced that over time more won't appear since it downloads a duplicate when one is already in the folder, so I'll keep my eye on it and let you know. 

Share this post


Link to post
Share on other sites
cayars

That's actually good because it might be downloading "the one" it wants after you remove them all.  Let's see what happens after a couple more days.

Share this post


Link to post
Share on other sites
cam1313

Nope, still downloading multiple subtitles and interestingly it's even downloading some of them in .txt format.

Share this post


Link to post
Share on other sites
cayars

Txt format?  See if you can find that file name in your server log file.  We should track that down to what provider it's coming from.

Share this post


Link to post
Share on other sites
AshranPewter
Posted (edited)

I'm having this issue too, there are 20+ subtitles in several movie folders now.

Though I have it set to download Chinese and English, only Chinese are being downloaded multiple times.

To note though, I'm in China Mainland behind the Great Firewall, so much of metadata stuff tends to not work, so the attempts in the log to cancel the task usually are attributed to http requests timing out.

edit:Additional question if it can be answered, would the logs show which metadata provider is having timeout requests? I would like to figure out which ones to disable so this stops happening. Emby doesn't seem to be able to properly handle timeout situations for library scans and won't cancel until you restart the server.

edit2: I do use addic7ed subtitles as well, maybe this has issues? is anyone else using this plug-in?

embyserver (1).txt

embyserver-63730454399.txt

Edited by AshranPewter
added question and another log

Share this post


Link to post
Share on other sites
cayars

You log is full of errors trying to fetch from Addic7ed.

Share this post


Link to post
Share on other sites
cam1313
On 7/16/2020 at 10:04 PM, cayars said:

Txt format?  See if you can find that file name in your server log file.  We should track that down to what provider it's coming from.

Sorry for the delayed response.  I've got several movies that have subtitles in .txt format.  Log attached.

embyserver.txt

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...