Converted media is being added back to the conversion queue (potential duplicates)

[milonedp 0]

I have set up a folder to cover media to a mobile format and place the files in the Original media folder, next to original files, I have selected the check box for "Automatically download new content" so that when I add moved they will be converted.

 

I have a folder running that is 90+ movies long, after each movie is done converting it then says transferring (even when there is no transfer step to occur, I'm only converting not downloading).

 

Then upon the next run of "Cloud & folder Sync" the movies are added again to the queue (as if the new "filename - mobile" is not going to be converted.

 

A few questions:

1. What is happening during this transfer step?

2. Will those "filename - mobile" fail out or create duplicates in a never ending loop?

3. I have "filename - 3D" files in the folders as well, will I get converted files for both the main and the 3d (I only want one)

4. Is there a different way to set this up? I want all movies converted once so we have a certain to take with us on mobile devices.

 

Thank you so much.

 

Sent from my Pixel 2 XL using Tapatalk

[Luke 37271]

Hi there, can you please go over an example? Thanks.

[milonedp 0]

I have a folder called Movies - Ori with 91 items in it, I set the folder to convert all media to

 

Sync to:  Original media folder, next to original files

Profile: Mobile

Quality: 4 Mbps

Download unwatched videos only - NOT SELECTED

Automatically download new contact - SELECTED

Item limit: BLANK

 

The Movie "Hotel Transylvania 2" was in the Items List with a status of Queued

It converted to "Hotel Transylvania 2 - mobile and the file shows on my NAS as "Hotel Transylvania 2 - mobile.mp4" - as expected

The item status changes to "Transferring" for a while, then goes away

Then at the next run of the task "Cloud & Folder Sync" Hotel Transylvania 2 is added to the end of the queue with a status of Queued again

 

I can't tell if it's seeing the new - mobile file and thinking it needs to also convert that one or what is happening.  To make matters even more strange, it's not 100% consistent, most files are being added back to the Items List but a few are not (which is the functionality that I'd actually expect)

 

Thanks

[Luke 37271]

What version of Emby Server?

[milonedp 0]

4.0.3.0 running on a Synology NAS

[milonedp 0]

I just checked one of the folders for one of the files that was re-populating the list and it created a "Filename - mobile - mobile.mp4"

 

My thought is to delete both the mobile and mobile - mobile files and run it again, but that is going to take considerable time (days) to let them all process then delete then let them run again.

 

Thoughts?

 

One thing I noticed is that there seems to be some correlation (maybe) that the files that this is happening with are files that I had previously downloaded to one of my daughter's chromebook.  (this didn't create the mp4 with the original files so I removed everything from all devices then started converting using my user name from my phone and PC), assuming that then in the future they could download any movie quickly because it was "pre-converted"

[Luke 37271]

Can you click into the detail screen of the web app for one of the "- mobile" copies, and copy the media info from the bottom of the screen and paste it here? thanks.

[milonedp 0]

Here is one that has a " - mobile" and a " - mobile - mobile"

 

 

Media Info

The Grinch (2018)

Video Title4K H264
CodecH264
AVCYes
ProfileHigh
Level51
Resolution3840x1606
Aspect ratio2.40:1
AnamorphicNo
InterlacedNo
Framerate23.976
Bitrate19,633 kbps
Bit depth8 bit
Pixel formatyuv420p
Ref frames1
NAL4

Audio TitleEng TRUEHD 7.1 Default
Languageeng
CodecTRUEHD
Layout7.1
Channels8 ch
Sample rate48,000 Hz
Bit depth24 bit
DefaultYes

Audio TitleEng DTS 7.1
Languageeng
CodecDTS
ProfileDTS-HD MA
Layout7.1
Channels8 ch
Sample rate48,000 Hz
Bit depth24 bit
DefaultNo

Audio TitleEng Dolby Digital 5.1
Languageeng
CodecAC3
Layout5.1
Channels6 ch
Bitrate448 kbps
Sample rate48,000 Hz
DefaultNo

Subtitle TitleEng Default
Languageeng
CodecSUBRIP
DefaultYes
ForcedNo
ExternalNo

Subtitle TitleEng
Languageeng
CodecPGSSUB
DefaultNo
ForcedNo
ExternalNo

Subtitle TitleSpa
Languagespa
CodecPGSSUB
DefaultNo
ForcedNo
ExternalNo

Subtitle TitleFre
Languagefre
CodecPGSSUB
DefaultNo
ForcedNo
ExternalNo

Embedded Image CodecMJPEG
Profile192
Level-99
Resolution213x120
Framerate90,000
Color spacebt470bg
Bit depth8 bit
Pixel formatyuvj444p
Ref frames1
DefaultNo

Embedded Image CodecMJPEG
Profile192
Level-99
Resolution600x882
Framerate90,000
Color spacebt470bg
Bit depth8 bit
Pixel formatyuvj444p
Ref frames1
DefaultNo

Embedded Image CodecMJPEG
Profile192
Level-99
Resolution1067x600
Framerate90,000
Color spacebt470bg
Bit depth8 bit
Pixel formatyuvj444p
Ref frames1
DefaultNo

Embedded Image CodecMJPEG
Profile192
Level-99
Resolution120x176
Framerate90,000
Color spacebt470bg
Bit depth8 bit
Pixel formatyuvj444p
Ref frames1
DefaultNo

Containermkv

Path/volume1/Family/Media/Movies/Movies - Ori/T/The Grinch (2018)/The Grinch (2018).mkv

Size12052 MB

 

mobile - mobile

Video Title720p H264
CodecH264
Codec tagavc1
AVCYes
ProfileHigh
Level41
Resolution1278x534
Aspect ratio2.40:1
AnamorphicNo
InterlacedNo
Framerate23.976
Bitrate3,610 kbps
Bit depth8 bit
Pixel formatyuv420p
Ref frames1
NAL4

Audio TitleEng AAC 5.1 Default
Languageeng
CodecAAC
Codec tagmp4a
ProfileLC
Layout5.1
Channels6 ch
Bitrate384 kbps
Sample rate44,100 Hz
DefaultYes

Containermp4

Path/volume1/Family/Media/Movies/Movies - Ori/T/The Grinch (2018)/The Grinch (2018) - mobile - mobile.mp4

Size2456 MB

 

mobile

Video Title720p H264
CodecH264
Codec tagavc1
AVCYes
ProfileHigh
Level41
Resolution1278x534
Aspect ratio2.40:1
AnamorphicNo
InterlacedNo
Framerate23.976
Bitrate3,610 kbps
Bit depth8 bit
Pixel formatyuv420p
Ref frames1
NAL4

Audio TitleEng AAC 5.1 Default
Languageeng
CodecAAC
Codec tagmp4a
ProfileLC
Layout5.1
Channels6 ch
Bitrate384 kbps
Sample rate44,100 Hz
DefaultYes

Containermp4

Path/volume1/Family/Media/Movies/Movies - Ori/T/The Grinch (2018)/The Grinch (2018) - mobile.mp4

Size2456 MB

Edited April 16, 2019 by milonedp

[Luke 37271]

Hmm, that is strange. Can you please provide the emby server log from when this happens? Thanks.

[milonedp 0]

Sorry if I missed this but how can I get a server log from a Synology NAS?

[Happy2Play 8361]

Sorry if I missed this but how can I get a server log from a Synology NAS?

 

The path is on the Dashboard in the Paths section.  But you can always go to Dashboard-Logs and open the log in the browser then right click on that page and save as.

 

 

@@Luke

Might be a good idea to add a download/export option on the logs page.

Edited April 18, 2019 by Happy2Play

[milonedp 0]

Happy2Play - THANK you so much, now for my stupid question, how do you find that on a NAS?  I tried going into the File Station and couldn't find it there, I tried a google search and all I could get was mapping to a NAS.  Sorry I'm such a newb on NASs.

[Luke 37271]

You can always go to Dashboard-Logs and open the log in the browser then right click on that page and save as.

[milonedp 0]

Okay so I just got home and I had the conversion running for the week while I was gone.  Some files created " - mobile - mobile - mobile - mobile.mp4" files.  I deleted those and tried to reload the convert program to see what I'd get however not knowing the impact I had changed the name of the folder that the files were in (I was using folders to trigger the conversion process) and now every movie including the " - mobile" queued up to convert.

 

I'm guessing there is a db somewhere that stores the info on which files were converted?  Is that correct?  If so where is it stored and can I change the reference from the old folder name to the new one, or do I need to delete all the " - mobile" files and start the conversion from start (there are over a hundred movies"

 

Is there any detailed documentation on how this all works so I can do some research on my own?

 

Thanks so much!

[milonedp 0]

In addition to above I now have a log for this issue.

 

Guardians of the Galaxy Vol 2 already has a - mobile file in it's folder, yet when I started a new convert on the main folder it queued it.  I let it convert again and it said it downloaded but there was no new file, only the original - mobile file and the main file are in the folder.  (Sometimes I get a - mobile - mobile, sometimes I see mobile - 1, and sometimes nothing new).  I go back and set the folder to convert again and the movie is again in the queue.  It's inconsistent, some movies never do this and others keep requeueing themselves.

 

Log attached

 

If I were to move everything out of my library including the - mobile files, delete all other files in the folders then return just the movie and the - mobile files and try to re-run everything will it try to convert both the original and the mobile again, or will it recognize the - mobile and not try to convert or is that not how this works?

Log.txt

[jon_ 23]

I had the same problem a while back:

 

https://emby.media/community/index.php?/topic/67602-minor-bug-moving-folder-with-transcoded-copies-creates-new-copies/

 

(although the moving folders is a red herring). It still does it now for some media files... 

 

For whatever reason, the transcoded copy isn't recognised as being 'right', so another copy is created... 

Edited April 28, 2019 by jon_

[milonedp 0]

It's still duplicating files. I have two main folders one for everyone (we have young daughters) and one that is restricted for the adults.

 

I got the everyone folder down to 15 movies that keep looping through. I tried moving these out of the library and folder getting rid of everything except the original .mkv running scan scan media, pulling it back into the "everyone" folder, running scan media then converting. All looked good for a bit but then on the following run of the convert media task the file was queued again.

 

Any thoughts on how to get past this? I have these 15 but over 90 on my other folder.

 

Thanks so much.

Edited April 30, 2019 by milonedp

[milonedp 0]

Any thoughts on this one?  I had as many as 5 duplicates of Return of the Jedi then I finally stopped it, some movies just keep looping themselves back into the queue and others complete their conversion and no further issues.

 

Would deleting the library from Emby, clearing out all files except the original movie and the - mobile movie, then recreating the library in Emby recognize the mobile files and not duplicate them again?  I have noticed that with all these mobile files the metadata manager and what shows under movies are now out of sync, so I'm wondering if the Library itself has become corrupted.

[Luke 37271]

I'm trying to chase this down, i just haven't been able to reproduce it yet. Thanks.

[milonedp 0]

Thanks. Let me know if I can provide anything else. I tried again on removing the offending movies (there are about 15) from the library by moving them to another folder on the drive. I moved one (Return of The Jedi) back in and now have 8 copies of the movie. What's odd is that they are all showing as different movies in Emby not combined like the rest of my library shows.

 

Sent from my Pixel 2 XL using Tapatalk

[Luke 37271]

Is there any common factor with the media that this is happening with? Are there mulitple audio tracks?

[milonedp 0]

I've tried to find some common pattern and I cannot.  Some have multiple audio tracks others do not.

[jrawsthorne 0]

I've had a similar issue. I used the mobile 8mps preset and sometimes the final bitrate is just above 8 e.g. 8050084. If I manually click convert on this item with the 8mps preset it tries to convert it again however if I set it to a custom bitrate of 8.1 it isn't converted again. It looks like something is going wrong when trying to limit the bitrate of the convertred file. It looks like the audio bitrate makes it go over.

[Luke 37271]

I've had a similar issue. I used the mobile 8mps preset and sometimes the final bitrate is just above 8 e.g. 8050084. If I manually click convert on this item with the 8mps preset it tries to convert it again however if I set it to a custom bitrate of 8.1 it isn't converted again. It looks like something is going wrong when trying to limit the bitrate of the convertred file. It looks like the audio bitrate makes it go over.

 

That's great info, thanks. We'll take a look at that.

[Q-Droid 670]

It appears that I am now running into this issue in 4.2.1.0, which I just upgraded to this morning from 4.1.1.0. As far as I can tell the sum of the bitrates for all of the streams in the converted file still exceed the limit given in the conversion setup. I use the mobile profile with a 5mbps limit. If the original has only one audio stream then the resulting converted files are fine. But if they have multiple audio then the converted file exceeds the limit and gets put back in the queue again.