Converted media is being added back to the conversion queue (potential duplicates)

[Luke 42077]

Can we look at an example? Thanks.

[Q-Droid 989]

This file was just converted and now it's in the queue to be converted again. I noticed earlier after the upgrade that many of the old converted movies in this library were re-queued and had multiple converted versions with names like <movie> - mobile - mobile.mp4. I thought it might have been a DB or metadata issue so I cleaned it up, removed old conversions and created a new Convert job after removing the old "download". But the new job is still re-queuing movies after conversion.

 

[Q-Droid 989]

Here's another one re-queued.

 

[Q-Droid 989]

More info: This is the kind of stuff I end up with in this library and the conversion is still running again for some of them.

 

-rw-r--r--+ 1 emby   video 3625260660 Aug 30 01:17 Elf (2003) - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video 3718641795 Aug 29 18:11 Elf (2003) - mobile.mp4

-rw-r--r--+ 1 emby   video 5397247128 Aug 30 01:36 Fantastic Beasts The Crimes of Grindelwald (2018) - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video 5397335697 Aug 29 19:24 Fantastic Beasts The Crimes of Grindelwald (2018) - mobile.mp4

-rw-r--r--+ 1 emby   video 4040397266 Aug 30 01:39 Halloween (2018) - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video 4074267893 Aug 29 19:40 Halloween (2018) - mobile.mp4

-rw-r--r--+ 1 emby   video 2721776676 Aug 30 07:53 Invader ZIM Enter the Florpus (2019) - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video 2721776676 Aug 29 21:18 Invader ZIM Enter the Florpus (2019) - mobile.mp4

-rw-r--r--+ 1 emby   video  5057274242 Aug 30 08:09 The Conjuring 2 (2016) - mobile - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video  5099215972 Aug 30 01:15 The Conjuring 2 (2016) - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video  5141642955 Aug 29 17:40 The Conjuring 2 (2016) - mobile.mp4

-rw-r--r--+ 1 emby   video  7359208986 Aug 30 07:58 The Right Stuff (1983) - mobile - mobile.mp4

-rw-r--r--+ 1 emby   video  7421268086 Aug 29 23:51 The Right Stuff (1983) - mobile.mp4

[Q-Droid 989]

Any ideas about this? As it is now the Convert feature is not usable for media with multiple audio tracks.

 

Let me know if more info is needed.

[milonedp 0]

Not that it's a fix but rather a work around, what I ended up doing was converting all the media in prep folder that had a convert job set to run against the folder then moving them both to the final library when I had time.

 

I made both folders accessible in the library so that the movie was there before conversion ran and I didn't have to move everything immediately.

 

Sent from my Pixel 2 XL using Tapatalk

[Q-Droid 989]

Not that it's a fix but rather a work around, what I ended up doing was converting all the media in prep folder that had a convert job set to run against the folder then moving them both to the final library when I had time.

 

I made both folders accessible in the library so that the movie was there before conversion ran and I didn't have to move everything immediately.

 

Sent from my Pixel 2 XL using Tapatalk

 

How does this prevent or work around re-queuing problem?

[milonedp 0]

How does this prevent or work around re-queuing problem?

 

Sorry, I didn't provide a good enough description.

 

Set Up:

Create a "Prep Folder"

Add that "Prep Folder" to the same library you normally use as an additional folder

Create a convert queue job that doesn't have the setting "automatically download new content". 

 

When you rip a new movie that needs to be converted:

1. Move the movie to the "Prep Folder"

2. Go to Scheduled Tasks:

2a. Run Scan Media Library

2b. Run Scan Convert Media

2c. Run Cloud and Folder Sync

NOTE:  I'm not sure you have to run all these

3. Go to Download and Sync

4. Find the Convert Job that is tied to your "Prep Folder"

5. Open it and hit save (you might have to toggle the "automatically download new content" hit save to get it started then come back in and turn it off I can't remember)

5a. You might need to go back to Scheduled Tasks and run Convert Media and Cloud and Folder Sync again, I can't remember)

6. Whenever you get a chance file these to their permanent folder

7. Go to Scheduled Tasks:

7a. Run Scan Media Library

 

NOTE:  As long as you include your "Prep Folder" in the Media Library the movies are available as soon as you run Scan Media and you don't need to move them right when the conversion happens, I've left some in there for weeks and the kids can happily go about watching them and downloading as needed.

 

The issue that I "think" is still happening is that if they try to download one of these movies that had multiple audio tracks the download doesn't pick up the previously converted file, it rather tries to re-convert it which then takes forever.

 

Let me know if that is more clear, it's pretty convoluted, but it doesn't take too much time to move the files and trigger the jobs...  A fix would be much better though!

[Q-Droid 989]

Thanks for the details. What I was trying to get was a better understanding of how a separate library path might help or prevent the re-queuing when the automatic library level convert is used. But it looks like the same thing with manual steps to prevent the auto conversion.

 

For the time being and until the actual problem is fixed I've opted to just convert each movie individually instead of doing so automatically for the library. In my case this particular library is more of a DVR destination so few movies are kept long term and the movie count is fairly low.

[Q-Droid 989]

@@Luke, has anyone on the team been able to reproduce or identify the cause?

[gleeballs 5]

@@Luke This is also happening to me. I am also getting -mobile-mobile files and all of them being readded into the convert queue.

 

Another quirk I find is that if we have already converted the file to mobile format/quality, when we choose to download/sync (too many names for the same thing IMO), why can we not download/sync the mobile version. We have to transcode the original version again and then download. Wasted time in both converting the file in the first place and then if we want an offline local copy we have to wait for the file to transcode anyway and then download/sync that

Edited October 10, 2019 by gleeballs

[Luke 42077]

@@Luke This is also happening to me. I am also getting -mobile-mobile files and all of them being readded into the convert queue.

Thanks for reporting. We are looking into this.

[Luke 42077]

 

 

Another quirk I find is that if we have already converted the file to mobile format/quality, when we choose to download/sync (too many names for the same thing IMO), why can we not download/sync the mobile version. We have to transcode the original version again and then download. Wasted time in both converting the file in the first place and then if we want an offline local copy we have to wait for the file to transcode anyway and then download/sync that

 

You can. Just select the appropriate quality that you want, and the download feature will use the most efficient version based on that selection.

[EmmTee 1]

I used convert for the first time the other day (prior to reading this thread) and ran into the same problem of multiple conversions. The first conversion I did was on an episode I had just downloaded and was run with these options:

• Original media folder, next to original files
• TV profile
• 4 Mbps quality

That worked flawlessly so I moved to run the same options as above on a folder with the additional options of:

• Automatically convert new content
• No item limit set

Although the folder only had 1 episode in it, it created 9 copies of the episode

• The Frozen Lake - tv.mkv
• The Frozen Lake - tv - tv.mkv
• The Frozen Lake - tv - tv (1).mkv
• The Frozen Lake - tv - tv - tv.mkv
• The Frozen Lake - tv - tv (1) - tv.mkv
• The Frozen Lake - tv - tv (1) - tv (1).mkv
• The Frozen Lake - tv - tv - tv - tv.mkv
• The Frozen Lake - tv - tv (1) - tv - tv.mkv
• The Frozen Lake - tv - tv (1) - tv (1) - tv.mkv

Thinking I must've done something wrong, I deleted all of them and ran the convert again and it created 1 copy as expected. So, I got bold and added another episode to the folder and it dutifully kicked off the program and created the following 2 copies:

• Los Angeles_5x1 - tv.mkv
• Los Angeles_5x1 - tv (1).mkv

The one monkey wrench I threw in there is that I believe on the 2nd round (when I restarted the first file), I also chose the "Convert unwatched videos only" option along with the "Automatically convert new content" option.

 

If you like, I can start over and include logs if that information is helpful. I had been converting everything separately but will quite enjoy this function when it's cleaner.

[Luke 42077]

@@EmmTee thanks for the info. Can you please provide the conversion log from an example? Thanks !

[Q-Droid 989]

@@Luke, does this mean you are now actively working on this? I've been doing manual conversions for movies since the problem started instead of allowing them automatically.

[EmmTee 1]

Thanks for checking in on this one. I had stopped using Convert and then forgot about it.

 

Tried again tonight with the following results:

1. Original media folder, next to original files
2. Profile: Mobile
3. 4 Mbps
4. limit of 2(there were 19 episodes in the folder)
5. This one just failed straight out

Ran it again with the following changes:

1. Original media folder, next to original files
2. Profile: Mobile
3. 4 Mbps
4. Unwatched only
5. limit of 1
6. Worked like a charm - only 1 file was converted and no duplicates

I will run a fuller set of scenarios tomorrow to see if the duplication issue remains.

server text log attached embyserver.txt

[EmmTee 1]

Ran a larger test yesterday that stretched into this morning but essentially, it is working famously. I do have one question about results that occurred in a different season that I don't recall asking for. Is that even possible?  Here's what I did using the same show/season as above:

1. Original media folder, next to original files
2. Profile: Mobile
3. 4 Mbps
4. Unwatched only
5. limit of 2
6. Worked perfectly as I already had 1 converted episode from yesterday so the limit I set of "2" meant that only 1 file was chosen and converted.

Ran another one with the following changes:

1. Original media folder, next to original files
2. Profile: Mobile
3. 4 Mbps
4. Unwatched only
5. limit of 4
6. Worked perfectly - only 2 files were converted totaling 4 episodes in that folder

Ran it again with the following changes:

1. Original media folder, next to original files
2. Profile: Mobile
3. 4 Mbps
4. Left 'Convert Unwatched Videos Only' unchecked
5. Did not set an item limit
6. Worked like a charm (i think) - I expected the other 15 episodes (S3) to convert and they did however, there were also conversions in the previous season (S2) and not all of the episodes were converted but nothing was touched in season 1

I did not get any duplicate conversions in any scenario and from what I've seen this weekend, it's working on episodes very well. I had the initial error but after that no errors and the weirdness of jumping episodes. Not sure what that's about. 

 

In any case, I've attached the server logs for the 16th, 17th, and 18th. I'm going to change my routine and put the Emby convert into my normal path and see what happens.

 

embyserver-518.txt

embyserver-517.txt

embyserver-516.txt

[Q-Droid 989]

Did any of your source items to be converted have multiple audio streams?

Edited May 18, 2020 by Q-Droid

[EmmTee 1]

All of the source items had one stream of audio. 

[EmmTee 1]

Ran another attempt this evening but this time on a movie. I used the following parameters:

1. Original Media folder, next to original files
2. Profile: Mobile
3. Quality: 4 Mbps

The movie had been extracted from an ISO using MakeMKV and had 1 video stream along with 3 audio streams. Audio were as follows:

1. ac3, surround 5.1, english
2. ac3, stereo, english
3. subtitle

The output was 4 files, all 15 MB in size (1:36 mins), and were given the familiar naming convention:

• Children of the Corn (1984) - tv.mkv
• Children of the Corn (1984) - tv (1).mkv
• Children of the Corn (1984) - tv (1) (1).mkv
• Children of the Corn (1984) - tv (1) (1) (1).mkv 

Each output contained the video stream, the 5.1 and stereo audio stream but not the subtitle stream. Log attached.

 

...Update-As of this morning (my time), the number of files has increased to (1) x 11 so I currently have 12 converted files all 15MB in size and potentially still multiplying. I'll leave the system as is and keep watching.

embyserver0613.txt

Edited June 14, 2020 by EmmTee

[Pockenfresse 3]

The bug is fixed in the version 4.6.0.6 beta. Can anyone confirm this?

Thank you, great job!

[Luke 42077]

Thanks for the feedback !

The fix is also in the 4.5.4 stable release that is beginning to roll out.