What is the purpose of the "sync" directory?

[rsvg 17]

I was just doing some house cleaning in my server and noticed my backups had increased in size, and so upon investigation I see that the sync directory is the large majority of the size.

Is this some type of cache?

What is it used for?

Can it be safely deleted?

Can you configure how often it is deleted/used?

Can it be configured to use a different storage device? I suppose I can just symlink it, but I am looking for best practices here.

Thanks!

Also, semi-unrelated, but I believe the "config-backup" directory was from a long time ago when I used the server backup plugin. I can safely delete that right? Thanks much!

 

[Luke 40114]

Quote

Also, semi-unrelated, but I believe the "config-backup" directory was from a long time ago when I used the server backup plugin. I can safely delete that right? Thanks much!

Hi, If you're no longer using it, then yes.

[Luke 40114]

The sync directory is used when you or your users create downloads or conversions, and the original file needs to be converted. The converted output will be stored in the sync directory until it is transferred to it's destination.

[rsvg 17]

6 minutes ago, Luke said:

Hi, If you're no longer using it, then yes.

Thanks much, correct I am not using it. Appreciate the info.

 

5 minutes ago, Luke said:

The sync directory is used when you or your users create downloads or conversions, and the original file needs to be converted. The converted output will be stored in the sync directory until it is transferred to it's destination.

Interesting. I will symlink to a larger drive, thanks. 

[rsvg 17]

Wanted to update this for anyone that happens upon it. 

After Luke provided the use of the directory I was able to navigate to ${emby}/web/index.html#!/conversions?mode=convert and see that I had a failed conversion of an entire tv series. As soon as I cancelled that the sync directory vanished in it's entirety. 

[J0ZAH 4]

So I'm trying to keep my docker server slim and the sync folder seems to be growing, I have checked the conversion tab in emby and it doesn't seem to be running any tasks. Is there a way to tell what it has converted and is there a task that cleans up the media? It was mentioned that it should be moving the converted files to a final destination but it doesn't seem to be finishing the transfer.

[Luke 40114]

On 4/20/2024 at 1:59 PM, J0ZAH said:

So I'm trying to keep my docker server slim and the sync folder seems to be growing, I have checked the conversion tab in emby and it doesn't seem to be running any tasks. Is there a way to tell what it has converted and is there a task that cleans up the media? It was mentioned that it should be moving the converted files to a final destination but it doesn't seem to be finishing the transfer.

HI, check both the conversions and also the downloads section in the server setup. You can delete any existing jobs you may have created on those two screens. When you delete the jobs it will clear the temp files from the sync directory. Please let us know if this helps. Thanks.

[embylad892746 31]

I also found my sync folder balooning in size. For linux users, scheduling a script to delete folders older than 'x' days should help:

# delete all folders older than 14 days
find /home/emby/sync/* -maxdepth 0 -mtime +14 -type d -exec rm -r {} \; 

can also do similar to clean out transcoding temp folder.

Edited May 9, 2024 by embylad892746

[Luke 40114]

5 hours ago, embylad892746 said:

I also found my sync folder balooning in size. For linux users, scheduling a script to delete folders older than 'x' days should help:

# delete all folders older than 14 days
find /home/emby/sync/* -maxdepth 0 -mtime +14 -type d -exec rm -r {} \; 

can also do similar to clean out transcoding temp folder.

HI, check both the conversions and also the downloads section in the server setup.

I would start there before just deleting the folders, because they may end up getting recreated if you still have jobs open.

[embylad892746 31]

6 hours ago, Luke said:

HI, check both the conversions and also the downloads section in the server setup.

I would start there before just deleting the folders, because they may end up getting recreated if you still have jobs open.

My Conversions page is empty.

My Downloads page has a mixture of "Downloaded" and "Ready to transfer".

If a download status is "Ready to transfer", these are the jobs with files taking up space in my sync folder?

 

Is there a more automated way I could "clean up" things in downloads / conversions folder using the API?

 

[Q-Droid 881]

The location is configurable, right?

[Luke 40114]

7 hours ago, embylad892746 said:

My Conversions page is empty.

My Downloads page has a mixture of "Downloaded" and "Ready to transfer".

If a download status is "Ready to transfer", these are the jobs with files taking up space in my sync folder?

 

Is there a more automated way I could "clean up" things in downloads / conversions folder using the API?

 

You don't have to manually clean them up. They already get cleaned up automatically when the jobs complete, so that's probably what needs looking at more than just deleting the files.

[embylad892746 31]

10 hours ago, Q-Droid said:

The location is configurable, right?

Hi thanks for this but I don’t see how this helps me. My sync folder is already mounted somewhere else. My problem is that my sync folder likes to balloon, the location of it is incidental. 

[embylad892746 31]

4 hours ago, Luke said:

You don't have to manually clean them up. They already get cleaned up automatically when the jobs complete, so that's probably what needs looking at more than just deleting the files.

That might be the theory but my reality is different. If there is anything I can do to helpful troubleshoot I’m open to suggestions. After a file is converted, does it stay present there until it’s actually downloaded? Could it be possible that users convert stuff then forgot / never fully complete the download, so it just stays there.

It would be nice if there was an option to set a time limit for the conversation/ download. E.g if users convert a file, they have ‘x’ days to download it otherwise the job gets cancelled and the corresponding file in sync folder gets deleted.

[Luke 40114]

Yes, it stays there until it is downloaded or transferred to it's destination.

Quote

It would be nice if there was an option to set a time limit for the conversation/ download. E.g if users convert a file, they have ‘x’ days to download it otherwise the job gets cancelled and the corresponding file in sync folder gets deleted.

A time limit would make sense, yes.

[benze 1]

Hi,

My config/sync folder is at 30G and yet:

- My Conversions tab show 0 items pending

- Downloads tab is 0 items

- Folder Sync tab is empty

Most recent file is dated 1month ago.

Can I safely just delete any files in here more than 30d old?

 

 

 

[Luke 40114]

7 minutes ago, benze said:

Hi,

My config/sync folder is at 30G and yet:

- My Conversions tab show 0 items pending

- Downloads tab is 0 items

- Folder Sync tab is empty

Most recent file is dated 1month ago.

Can I safely just delete any files in here more than 30d old?

 

 

 

Yes, then keep an eye out of it happens again with new downloads or conversions. If it does then the server could potentially be being denied access to clean up those files.

[Noodlewasser 66]

On 5/11/2024 at 12:03 AM, Luke said:

Yes, it stays there until it is downloaded or transferred to it's destination.

A time limit would make sense, yes.

What's the status on this? I just cleaned up a 507 GB sync folder with some files in it older than a year. It would be great if it would be cleaned automatically after 2 or 4 weeks.

[Luke 40114]

6 hours ago, Noodlewasser said:

What's the status on this? I just cleaned up a 507 GB sync folder with some files in it older than a year. It would be great if it would be cleaned automatically after 2 or 4 weeks.

Hi, this hasn't been done yet, but yes it is something that we should look at. In the meantime I would just delete the really old ones manually.

[chowbok 113]

22 hours ago, Noodlewasser said:

What's the status on this? I just cleaned up a 507 GB sync folder with some files in it older than a year. It would be great if it would be cleaned automatically after 2 or 4 weeks.

You could always schedule "find /path/to/syncdir -type f -mtime 30 -delete" as a daily task in your crontab or as a systemd timer.