Using Tdarr, a few questions

[podonnell 47]

I am starting to use Tdarr to remove extra subtitles and audio tracks that I don't want in my files.

The way this works is a file is added to my Emby library folders, but around the same time Tdarr and Emby see the file -- so the real-time monitoring is kicking in while Tdarr is reprocessing the file and ultimately copying in a 'new' file after its done.

How will Emby handle this? Will it realize that is a different file, even though its named the same? Or will I need to run a manual library scan to detect that?
Is there any other processing Emby is doing on 'new' files that I should be concerned about that might be happening twice now? For example, grabbing metadata and updating it or otherwise?

Edited August 23, 2024 by podonnell

[Neminem 1518]

That really depends on you setup.

Is Tdarr is able to do atomic move from transcoding folder to media storage ?

If yes Emby will see the new file as new, but if same filename it should keep metadata.

In worst case Tdarr tries to copy the file, awsomemovie.fdjhsgfjhsg.mkv temp to folder

Emby will see that temp copy file as a completely new file try to fetch metadata.

When Tdarr is done its copy it will rename the file to original file.

 

But then emby thinks the file is gone, and will delete the movie and then see the movie again and get metadata.

 

Your best option is to :

1) make sure Tdarr can do the atomic move.

2) If not turn off RTM.

 

Edited August 23, 2024 by JayceDK

[Happy2Play 9780]

6 minutes ago, JayceDK said:

That really depends on you setup.

Is Tdarr is able to do atomic move from transcoding folder to media storage ?

If yes Emby will see the new file as new, but if same filename it should keep metadata.

In worst case Tdarr tries to copy the file, awsomemovie.fdjhsgfjhsg.mkv temp to folder

Emby will see that temp copy file as a completely new file try to fetch metadata.

When Tdarr is done its copy it will rename the file to original file.

 

But then emby thinks the file is gone, and will delete the movie and then see the movie again and get metadata.

 

Your best option is to :

1) make sure Tdarr can do the atomic move.

2) If not turn off RTM.

 

Do you know if Tdarr keeps same datetime stamp as over here it would appear Emby never saw the update and required the media to be touched.

 

[Dickydodah! 154]

I use TDARR for exactly this reason (removing subtitles and audio tracks). I also downmix to stereo as I don't have a surround system and I was sometimes getting Emby transcoding which I don't want as my server is low powered. I've never had any real issues that you are describing but there are settings in TDARR that might help. I have RTM on in Emby and in TDARR I also have Folder Watch on (their name for RTM) There is an option to hold files after scanning and I have this set to 300 seconds which is long enough to let Emby do it's thing. This has been working flawlessly for a couple of years now and I'm still using classic plugins as I don't want any more functionality in TDARR and can't be bothered to learn their new Flows system 

[podonnell 47]

1 hour ago, JayceDK said:

Your best option is to :

1) make sure Tdarr can do the atomic move.

2) If not turn off RTM.

 

I was thinking about this, basically making Tdarr watch a folder and then do its thing, with the output_folder being the Emby watched folders. However this would mean I need to set up one of these for each library as it seems Tdarr only has a single output_folder from what I saw. Keeping this in mind though...

47 minutes ago, Dickydodah! said:

There is an option to hold files after scanning and I have this set to 300 seconds which is long enough to let Emby do it's thing. This has been working flawlessly for a couple of years now and I'm still using classic plugins as I don't want any more functionality in TDARR and can't be bothered to learn their new Flows system 

Where is the option to hold files after scanning? And so I have this right, this means...

1) You get file.mkv in your emby folder
2) Tdarr starts processing file.mkv in a separate temp folder
3) Tdarr finishes, and now copies and replaces file.mkv with the newly processed file

Without a 300 second pause between #2 and #3, it could delete the file that Emby is working with, and cause issues I reckon?

[Neminem 1518]

1 hour ago, Happy2Play said:

Do you know if Tdarr keeps same datetime stamp as over here it would appear Emby never saw the update and required the media to be touched.

There is a plugin that can do that.

[Neminem 1518]

8 minutes ago, podonnell said:

I was thinking about this, basically making Tdarr watch a folder and then do its thing, with the output_folder being the Emby watched folders. However this would mean I need to set up one of these for each library as it seems Tdarr only has a single output_folder from what I saw. Keeping this in mind though...

No, you can setup multiple and no, you can setup multiple.

In Tdarr you can setup multiple library's to watch.

And under each library you can set export / output folders.

c:\Incomming\TV -> c:\Emby\tv

c:\Incomming\Movies -> c:\Emby\movies

Edited August 23, 2024 by JayceDK

[Dickydodah! 154]

9 minutes ago, podonnell said:

Where is the option to hold files after scanning? And so I have this right, this means...

1) You get file.mkv in your emby folder
2) Tdarr starts processing file.mkv in a separate temp folder
3) Tdarr finishes, and now copies and replaces file.mkv with the newly processed file

Without a 300 second pause between #2 and #3, it could delete the file that Emby is working with, and cause issues I reckon?

On the libraries tab select the library and the option is at the end of the source page.

The 300 second pause is actually between 1 and 2 as Emby and TDARR bothe "see" the new file at the same time. It really wouldn't matter if TDARR changed the file whilst Emby was looking at it as Emby would see the new file and rescan it but it is untidy so I delayed TDARR.

[Dickydodah! 154]

12 minutes ago, JayceDK said:

There is a plugin that can do that.

For some reason that plugin never worked for me but it didn't really matter as Emby just sees the new file exactly as if a better file had been downloaded by one of the ARRs

[podonnell 47]

On 8/23/2024 at 3:29 PM, Dickydodah! said:

On the libraries tab select the library and the option is at the end of the source page.

The 300 second pause is actually between 1 and 2 as Emby and TDARR bothe "see" the new file at the same time. It really wouldn't matter if TDARR changed the file whilst Emby was looking at it as Emby would see the new file and rescan it but it is untidy so I delayed TDARR.

I went to do this and actually realized it's Tdarr you are delaying, and not Emby? So you let Emby scan the file and whatever processing, then after 300 seconds you allow Tdarr out of the gate to do its thing?

My initial thought here was to prevent Emby from scanning until Tdarr finished and supplied the new file.

Wouldn't this be what happens now?

1) Emby sees the file, runs processing
2) After 300 seconds, Tdarr starts its transcoding/processing
3) Once complete, Tdarr starts copying the transcoded file into the emby folder
4) Possibly, emby will now see this copy and start running its processes on it
5) The copy completes, and the original file is deleted, temp file is renamed
6) Emby is maybe confused here?? It was processing on a temp file that is now renamed to the original file name
7) Emby starts over processing on the finalized Tdarr file

Another thing to mention - I have my transcode cache on a separate drive(an ssd) than where I store my media(an hdd). So my copy might take longer than normal. Is this recommended or no?

[Happy2Play 9780]

Is TDarr processing the media in the same media folder or a folder outside the library?  If the same folder then Emby RTM will be somewhat useless as processing original and converting media continuously and or multiple time per movie.

As I see it Emby should process the original

TDarr does its conversion outside Emby, then replaces original (depending on how long the copy/move/replace original takes and the method used RTM may trigger more than one event)

RTM should see the change and reprocess the converted version.

[podonnell 47]

2 minutes ago, Happy2Play said:

Is TDarr processing the media in the same media folder or a folder outside the library?  If the same folder then Emby RTM will be somewhat useless as processing original and converting media continuously and or multiple time per movie.

As I see it Emby should process the original

TDarr does its conversion outside Emby, then replaces original (depending on how long the copy/move/replace original takes and the method used RTM may trigger more than one event)

RTM should see the change and reprocess the converted version.

Outside the library -- I have a transcode cache folder on my SSD. However my library folder is on HDD so I'm unsure if the transfer time is worth the cache folder. Nevertheless...

Tdarr would indeed then be processing outside Emby, but it has a somewhat lengthy copy for 4k files, of which I think Emby would start processing before the copy is done.

My primary questions are:

1) Is all the RTM processing wasteful, or not a big deal? I'd love to have Emby process a single file, but if its not a big deal then that's fine
2) Should I keep my tdarr transcoding on my SSD? If I moved my transcode cache to my HDD, would all the processing on that disk potentially disrupt ongoing streams / cause disk thrashing? 

[Happy2Play 9780]

1) Sort of a personal preference as Emby see the media as I add it vs relying on next library scan.  But obviously the media is going to be process more than once per the work flow of Original in Emby, then TDarr conversion, replace original process.  So a minimum of Emby processing the media twice.

6 minutes ago, podonnell said:

but it has a somewhat lengthy copy for 4k files, of which I think Emby would start processing before the copy is done.

May need to look at extending the RTM delay so then as default in the system.xml.

<LibraryMonitorDelaySeconds>90</LibraryMonitorDelaySeconds>

2) potentially yes doing the conversion on the same library drives could affect other operations as you are already read the original content.

[Dickydodah! 154]

It looks like you have a very similar setup to me. I have everything running on the same machine. SABnzbd downloads to my SSD then unpacks to my Media drive which is a 24TB DrivePool RAID 0. Sonarr/Radarr then renames it and moves the file to the Emby Library folder on the DrivePool drive. This is almost immediate as it is a move on the same drive. TDARR then "sees" the file and waits 300 seconds (or however long you want) before transcoding the file onto the SSD. once finished TDARR moves the temp file to the DrivePool and then deletes the origional followed by a rename to replace the origional file. If Emby actually responds to the new temp file before it's deleted in the replace operation doesn't seem to matter as it then cancels the operation on the temp file and "sees" the new version of the origional file and processes that. Emby doesn't seem to react quite as fast as you may think and most of the time doesn't even "see" the TDARR temp file. This is all on a very old machine that can't even do any real time transcoding so unless you are like me and enjoy proving Microsoft wrong and are running W11 on a HP ProLiant N54L G7 MicroServer you should have no problems. BTW that's not the oldest machine I run W11 on 

[Dickydodah! 154]

I see that there has been a couple of replies whilst I was single finger typing  You could do it the way I have described and yes there is a possibility of minor congestion when moving a 4k file. however you could just put the TDARR cache on the same drive and then move the file which is instantaneous as there is no disk write involved, just a change in the adressing of the file. I think that you may be overthinking the problem (something I do a lot) and i suggest that you try it as i think you will be plesantly surprised how well it all works.

[podonnell 47]

21 minutes ago, Dickydodah! said:

I see that there has been a couple of replies whilst I was single finger typing  You could do it the way I have described and yes there is a possibility of minor congestion when moving a 4k file. however you could just put the TDARR cache on the same drive and then move the file which is instantaneous as there is no disk write involved, just a change in the adressing of the file. I think that you may be overthinking the problem (something I do a lot) and i suggest that you try it as i think you will be plesantly surprised how well it all works.

Yeah I think we are the same person. I'm also on a very old machine, with DrivePool, and love to overthink.

I think perhaps moving the Tdarr transcode to the HDD instead as the copying back and forth is probably not worth it.
My biggest question would then be if you have ever seen playback issues during a Tdarr transcode? That's my only reason for moving it to SSD otherwise.

Adding to the RTM delay could be nice too but then I feel like I'd be pushing off seeing other files as quickly as I could.

[Neminem 1518]

There is an optional node schedule you could tinker with.

That way you schedule transcoding to your of hours.

Having it only run when you sleep.

[Dickydodah! 154]

The only issues I have seen have been non issues as I only really saw them in the logs. On the very rare occasion that something is watched as soon as it is added to Emby TDARR starts a transcode but cannot overwrite the origional and you get a file in use error. I believe that it trys again later and eventually succeeds. On older versions of TDARR I very rarely found that Emby showed two versions of the same media as the RTM has found the converted file that couldn't overwrite the origional as it was in use. A simple windows search for "TDARR" in your media folder will easily find these orphaned files and I think they are also logged in TDARR as a fail but I'm not sure of that.

I've just done a search for "TDARR" in my TV folder and it found none 

Edited August 28, 2024 by Dickydodah!