Jump to content

Unraid Emby server docker seems to be filling up docker image.


Go to solution Solved by Lessaj,

Recommended Posts

Posted (edited)

hmm   I seem to have this issue. also on UNRAID  (been using emby since MediaBrowser3, UNRAID for 8 years)

every 24 hours there is 50+ GB of writeable files in my docker image.
If i "force update" the docker container, or update to new version, writeable goes to 0

It turns out I was aware of this years ago and set a script to re-download the docker everynight.
forgot all about it,  found the script last month and forgot why it existed.

docker file kept filling up.
found this.  again.
trying to fix this time.


image.png.bd9bbcfdf54f9cacd4175f2a885ea03e.png


I reconfigured my docker container 2 days ago with the recommended variables ports and paths from the github instructions.

Changed the UID PID and GIDLIST to values obtained using instructions on github,  values used to be 100
No change.

image.png.e1e6ec5ae5dd0559a38225dabfd25f26.png

image.png.b03319757ad7c6a5a62583944356be55.png

image.png.9efe7fd7b6a51da294f16de81d00195b.png

 

also ran a recursive CHmod 777 on every file and folder of /emby data and /tv and /movies (found under /mnt/user)

my "emby data" folder on the server has 3 folders inside.

metadata

transcoding-temp

cache 

all 3 folders have modified folders/files less than a day old. 

BUT...  transcoding-temp  folder is full of 238 empty folders.  
so it is able to create folders

started playback, changed resolution.

refresh folder,  new folder in trans-temp, full of transcoding files.  it can create files,

stopped playback, files gone.  it can delete files  /./  but not empty folders?

k that is working.

out of ideas.

Edited by trusselo
Posted (edited)

first time ive run into an issue inside a docker.

just figured out what to do..

opened up a terminal in the EmbyServer container

/ # ls
bin       dev       etc       init      libexec   mnt       root      sys       tmp       var
config    embydata  home      lib       licenses  proc      share     system    usr

/ # du ?

Usage: du [-aHLdclsxhmk] [FILE]...

Summarize disk space used for FILEs (or directories)

        -a      Show file sizes too
        -b      Apparent size (including holes)
        -L      Follow all symlinks
        -H      Follow symlinks on command line
        -d N    Limit output to directories (and files with -a) of depth < N
        -c      Show grand total
        -l      Count sizes many times if hard linked
        -s      Display only a total for each argument
        -x      Skip directories on different filesystems
        -h      Sizes in human readable format (e.g., 1K 243M 2G)
        -m      Sizes in megabytes
        -k      Sizes in kilobytes (default)
/ # du -hasxc -d 1
15.3M   ./bin
460.0K  ./etc
0       ./home
0       ./root
50.9G   ./tmp
7.4M    ./usr
184.0K  ./var
511.1M  ./lib
384.0K  ./licenses
1.8M    ./share
0       ./.gitkeep
177.6M  ./system
4.0K    ./init
108.0K  ./libexec
0       ./.dockerenv
51.6G   .
51.6G   total
/ # 

/tmp folder

/ # cd /tmp
~ # ls
100                                     137                                     clr-debug-pipe-212-188679801-in
102                                     138                                     clr-debug-pipe-212-188679801-out
136                                     71                                      dotnet-diagnostic-212-188679801-socket
~ # cd /tmp
~ # du -hasxc -d 3
0       ./dotnet-diagnostic-212-188679801-socket
0       ./clr-debug-pipe-212-188679801-in
0       ./clr-debug-pipe-212-188679801-out
4.0K    ./.dotnet/shm/global
4.0K    ./.dotnet/shm
4.0K    ./.dotnet
4.0K    ./.nv/ComputeCache/index
4.1M    ./.nv/ComputeCache/b
4.0K    ./.nv/ComputeCache/e
164.0K  ./.nv/ComputeCache/8
4.0K    ./.nv/ComputeCache/1
4.3M    ./.nv/ComputeCache
4.3M    ./.nv
3.0G    ./71/176/2372a1c0-1617-48b6-9a93-b72c50ec75d4.mkv
3.0G    ./71/176
3.0G    ./71/177/741a0202-5a2d-480f-852d-25b5f09fbac5.mkv
3.0G    ./71/177
3.6G    ./71/178/98ae737b-de66-4d18-b912-360dadbfe34c.mkv
3.6G    ./71/178
3.8G    ./71/179/ed055f96-a024-4e5f-b317-470217267f1e.mkv
3.8G    ./71/179
3.8G    ./71/369/108a3b6c-436d-48e4-896d-4d1006e9b662.mkv
3.8G    ./71/369
3.0G    ./71/397/f2ac1abe-9036-4ce7-ac3b-1a0c633fa040.mkv
3.0G    ./71/397
3.0G    ./71/398/7955d2ca-9a70-426a-9b2a-4649768262d8.mkv
3.0G    ./71/398
3.9G    ./71/399/7a86e397-f6c4-41c8-89b0-1d607dcacc0f.mkv
3.9G    ./71/399
3.5G    ./71/400/23ee51ea-f1bb-4089-bd86-30924309412c.mkv
3.5G    ./71/400
30.6G   ./71
3.9G    ./100/280/ca412039-4c4c-432a-8199-08297e9b0f68.mkv
3.9G    ./100/280
3.9G    ./100
3.9G    ./102/282/c91aba1a-3a7b-4417-a08d-9c7060c73915.mkv
3.9G    ./102/282
3.9G    ./102
4.2G    ./136/411/3d86d28d-de27-4811-a91d-db95771cf5c8.mkv
4.2G    ./136/411
4.2G    ./136
4.2G    ./137/412/8e011bce-6364-444e-88de-5b58e06ba19b.mkv
4.2G    ./137/412
4.2G    ./137
4.2G    ./138/413/53b9c700-e023-4c95-af30-5c74193f2309.mkv
4.2G    ./138/413
4.2G    ./138
4.0K    ./.ash_history
50.9G   .
50.9G   total

tmp folder is the culprit 

Edited by trusselo
Posted (edited)

Remember: as tested earlier... files/folders DO create and delete from transcoding-temp when i start / stop a transcoding stream

looking in emby transcode logs, I do not see matching filenames... but here are the most recent server and transcoding logs.

they appear to show the transcoding temp files going into /embydata/transcoding-temp/ like they should...

embyserver.txt embyserver-63899171956.txt embyserver-63899193600.txt hardware_detection-63899171962.txt ffmpeg-remux-013602f3-18b5-4055-86c4-ebfe147696c2_1.txt ffmpeg-transcode-9b304a49-f3a9-4970-9e79-1360339c1d54_1.txt ffmpeg-transcode-049d5982-b713-46ed-93dd-9300fe21acf5_1.txt ffmpeg-transcode-801cb5c6-d8f4-46b3-ba61-be836a9e5470_1.txt ffmpeg-transcode-db30eda1-64b9-495f-86bf-258c48a2d8b9_1.txt

Edited by trusselo
Posted

What is tmp? The server does not have any folders with that name.

Posted
43 minutes ago, Luke said:

What is tmp? The server does not have any folders with that name.

I have no idea.
I did not create it.

It exists within the docker container.

this  is my docker config

I do not define a tmp folder/volume
 

docker run
  -d
  --name='EmbyServer'
  --net='host'
  --pids-limit 2048
  -e TZ="America/Denver"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="BIG-Z"
  -e HOST_CONTAINERNAME="EmbyServer"
  -e 'TCP_PORT_8096'='8096'
  -e 'UID'='0'
  -e 'GID'='0'
  -e 'GIDLIST'='18'
  -e 'NVIDIA_DRIVER_CAPABILITIES'='all'
  -e 'NVIDIA_VISIBLE_DEVICES'='GPU-bf2b870d-62dc-30c6-0c3d-0727ab981fb3'
  -l net.unraid.docker.managed=dockerman
  -l net.unraid.docker.webui='http://[IP]:[PORT:8096]/'
  -l net.unraid.docker.icon='https://raw.githubusercontent.com/MediaBrowser/Emby.Resources/master/images/Logos/logoicon.png'
  -v '/mnt/user/':'/mnt':'rw'
  -v '/mnt/user/emby data/':'/embydata':'rw'
  -v '/mnt/user/appdata/EmbyServer':'/config':'rw'
  --device='/dev/dri'
  --runtime=nvidia
  --restart unless-stopped 'emby/embyserver:latest'

 

Posted (edited)

 

 

53 minutes ago, Luke said:

What is tmp? The server does not have any folders with that name.

after restarting the docker embyserver
running UNRAID "container size" 

Name                                            Container                Writable                 Log

EmbyServer                                      739 MB                   15.5 kB                  93.0 kB

 

back into the terminal

/ # ls
bin       dev       etc       init      libexec   mnt       root      sys       tmp       var
config    embydata  home      lib       licenses  proc      share     system    usr
/ # cd tmp
~ # ls
clr-debug-pipe-215-197447402-in         clr-debug-pipe-215-197447402-out        dotnet-diagnostic-215-197447402-socket
~ # 

files gone, they will be back tomorrow.

 

And i confirmed i am running the official dockerhub emby/embyserver not a fork or linuxserver.io version

Edited by trusselo
  • Solution
Posted

Since these files are MKV it looks like they are conversions for download jobs to me. What path do you have set under Conversion -> Settings? Are there any download jobs set up? Conversions and Downloads are here in the sidebar.

image.png.68ac3c85956a6aa3e9bb647a2224e1f4.png

  • Like 2
Posted (edited)
16 minutes ago, Lessaj said:

Since these files are MKV it looks like they are conversions for download jobs to me. What path do you have set under Conversion -> Settings? Are there any download jobs set up? Conversions and Downloads are here in the sidebar.

image.png.68ac3c85956a6aa3e9bb647a2224e1f4.png

 

image.png

never thought of conversions.  NAILED IT!

Edited by trusselo
Posted

Mystery solved! It does not log this path in the log even if you configure it like some of the other paths, I checked.

Posted
17 minutes ago, Lessaj said:

Since these files are MKV it looks like they are conversions for download jobs to me. What path do you have set under Conversion -> Settings? Are there any download jobs set up? Conversions and Downloads are here in the sidebar.

 

but thats half the battle. 

why are they not clearing?

Posted (edited)
1 minute ago, Lessaj said:

Mystery solved! It does not log this path in the log even if you configure it like some of the other paths, I checked.

so i changed the path.
we will see what happens. if they build up or clear properly

image.png.a3736cda6ee363a47889d85bdb974f70.png

Edited by trusselo
Posted
1 minute ago, trusselo said:

but thats half the battle. 

why are they not clearing?

I know that if there's a Download job that's stuck in Ready to Transfer that it will stay there, that was why I asked if there were any. I have some right now, and there's folders related to them, but they're empty. The status of all the jobs is Downloaded, and I think at that point the files are removed. The folders probably disappear when the job is removed, but I don't want to remove anyone's jobs so I'd need to test myself.

Posted
2 minutes ago, Lessaj said:

I know that if there's a Download job that's stuck in Ready to Transfer that it will stay there, that was why I asked if there were any. I have some right now, and there's folders related to them, but they're empty. The status of all the jobs is Downloaded, and I think at that point the files are removed. The folders probably disappear when the job is removed, but I don't want to remove anyone's jobs so I'd need to test myself.

I get 50 GB in 24 hours.

My emby server has been running since MediaBrowser days, pretty sure when i switched from a Windows server to Docker, I just copied my emby settings file over.
This "MIGHT" be where the /tmp  got configured....?
or it is a leftover from previous build of the docker.
I am not the only person.  I resurected a dead thread with the same issue. then luke moved me to new post.

might want to code a check into the server docker to look for /tmp folders & move to transcoding-temp/conversions

Posted (edited)
37 minutes ago, Lessaj said:

I know that if there's a Download job that's stuck in Ready to Transfer that it will stay there, that was why I asked if there were any. I have some right now, and there's folders related to them, but they're empty. The status of all the jobs is Downloaded, and I think at that point the files are removed. The folders probably disappear when the job is removed, but I don't want to remove anyone's jobs so I'd need to test myself.

better idea.  on server dashboard.  show emby data folders, their sizes, and their configured paths plus an icon beside each to take you to the corresponding page to change path/settings.

1 place to oversee/manage all folders.

cache                               /embydata/cache                        40GB
transcoding-temp          /embydata/transcoding-temp    0kb
metadata                        /embydata/metadata                   60MB
conversions                   /tmp/                                               50GB
logs                                 /logs/                                               2MB
 

with a CLEAR/DELETE button beside each

just realized emby does not have option to move logs location...

now i want to map my /logs to a volume outside the docker incase i cant start the container....  not that it has happened in 15 years

Edited by trusselo
Posted
40 minutes ago, trusselo said:

but thats half the battle. 

why are they not clearing?

Either the output hasn’t reached its destination yet or the server is being denied access to delete from that folder.

Posted
32 minutes ago, trusselo said:

I get 50 GB in 24 hours.

My emby server has been running since MediaBrowser days, pretty sure when i switched from a Windows server to Docker, I just copied my emby settings file over.
This "MIGHT" be where the /tmp  got configured....?
or it is a leftover from previous build of the docker.
I am not the only person.  I resurected a dead thread with the same issue. then luke moved me to new post.

might want to code a check into the server docker to look for /tmp folders & move to transcoding-temp/conversions

If the size is always pretty consistent after restarting the container (which clears out /tmp) that suggests the same conversions keep happening because the files keep disappearing. Since you've changed it to a disk backed path it shouldn't be an issue anymore.

Posted

By the way I just tried on my system and yes the MKV disappeared after it was downloaded, so check your download jobs if any are stuck as ready to transfer, maybe clean up some old jobs.

While converting
./
./247/1232
./247/1232/af60cd61-7781-407e-badf-ee33121d7fdf.mkv

After download finished
./
./247

 

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...