Issue downloading some content to Android - "Ready to Transfer"

[98DAM 0]

I have started to have what seem to be random episodes or movies which are stuck at "Ready to Transfer".   I enabled debug logging on the client and server and nothing stands out except all of the media which have an issue show up as "Ready to sync in the logs".  I am now up to 13 episodes in this state. Here is the snippet from the Server logs with the media title edited.

2024-09-06 22:47:22.659 Debug Server: http/1.1 GET http://x.x.x.x:8096/emby/system/info/public?format=json. Source Ip: host5, UserAgent: okhttp/4.11.0
2024-09-06 22:47:22.659 Debug Server: http/1.1 Response 200 to host5. Time: 0ms. GET http://x.x.x.x:8096/emby/system/info/public?format=json
2024-09-06 22:47:22.863 Debug Server: http/1.1 GET http://x.x.x.x:8096/emby/Sync/Items/Ready?TargetId=4d1fc7329d770fcf. Source Ip: host5, UserAgent: okhttp/4.11.0
2024-09-06 22:47:22.977 Debug SqliteItemRepository: GetitemById Episode 449846 /mnt/TV/Show1/Season 1/Show1 - S01E05.mkv
2024-09-06 22:47:22.978 Debug SqliteItemRepository: GetitemById Episode 465057 /mnt/TV/Show1/Season 1/Show1 - S01E06.mkv
2024-09-06 22:47:22.981 Debug SqliteItemRepository: GetitemById Episode 224870 /mnt/TV/Show2/Season 1/Show2 - S01E06.mkv
2024-09-06 22:47:22.982 Debug SqliteItemRepository: GetitemById Episode 462470 /mnt/TV/Show3/Season 1/Show3 - S01E03.mkv
2024-09-06 22:47:22.983 Debug SqliteItemRepository: GetitemById Episode 224866 /mnt/TV/Show2/Season 1/Show2 - S01E08.mkv
2024-09-06 22:47:22.984 Debug SqliteItemRepository: GetitemById Episode 224865 /mnt/TV/Show2/Season 1/Show2 - S01E09.mkv
2024-09-06 22:47:22.986 Debug SqliteItemRepository: GetitemById Episode 517571 /mnt/TV/Show4/Season 1/Show4 - S01E04.mkv
2024-09-06 22:47:22.987 Debug SqliteItemRepository: GetitemById Episode 557290 /mnt/TV/Show4/Season 1/Show4 - S01E06.mkv
2024-09-06 22:47:22.988 Debug SqliteItemRepository: GetitemById Episode 605155 /mnt/TV/Show4/Season 1/Show4 - S01E09.mkv
2024-09-06 22:47:22.989 Debug SqliteItemRepository: GetitemById Episode 605156 /mnt/TV/Show5/Season 1/Show5 - S01E04.mkv
2024-09-06 22:47:22.990 Debug SqliteItemRepository: GetitemById Episode 621982 /mnt/TV/Show5/Season 1/Show5 - S01E05.mkv
2024-09-06 22:47:22.992 Debug SqliteItemRepository: GetitemById Episode 621985 /mnt/TV/Show4/Season 1/Show4 - S01E10.mkv
2024-09-06 22:47:22.993 Debug SqliteItemRepository: GetitemById Episode 627059 /mnt/TV/Show6/Season 2/Show6 - S02E04.mkv
2024-09-06 22:47:22.994 Debug Sync: Returning 13 ready sync items for targetId 1

A few items I have tried:

1) The media all plays fine manually

2) The media downloaded fine on my iPad

3) I tried transcoding the media to various formats and even if I transcode or copy a new version of an episode it still won't download

4) I changed the location to external so I could see the files and it is created zero byte copies of the episodes it cannot download.  If I delete that zero byte file it doesn't change anything

5) The permissions of the file on disk are the same as what I have on other files with the user account running Emby has rw to it and the folder

6) I am not transcoding the files but I did try transcoding them.

7) Upgraded to the beta version of Emby server 4.9.30

Tried changing the option from unwatched only based on another thread

9) I installed the latest apk from the Emby Github page.

10) Renamed the files just in case a special character was messing it up

 

 

The logs from the Android client doesn't seem to show anything but here is what shows up:

Fri Sep 06 19:17:48 EDT 2024 CameraUploadWorker: Start executing job.
Fri Sep 06 19:17:48 EDT 2024 Found 0 server(s) for camera upload
Fri Sep 06 19:17:48 EDT 2024 CameraUploadWorker: Exit SUCCESS
Fri Sep 06 19:17:48 EDT 2024 AppSyncWorker: Start executing job.
Fri Sep 06 19:17:48 EDT 2024 ServerSync: Starting sync for server: 58f71751de0f407fa614d45c9248388a 'home'
Fri Sep 06 19:17:48 EDT 2024 MediaSync: Begin checkLocalFileExistence
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin reportOfflineActions
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewMedia
Fri Sep 06 19:17:49 EDT 2024 Getting response stream from http://x.x.x.x:8096/emby/Sync/Items/Ready?TargetId=4d1fc7329d770fcf
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getReadySyncItems returned 13 items
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin getNewItem
Fri Sep 06 19:17:49 EDT 2024 MediaSync: getLibraryItem found existing item
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin syncData
Fri Sep 06 19:17:49 EDT 2024 Getting response stream from http://x.x.x.x:8096/emby/Sync/data
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Exit syncData
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Begin afterSyncData
Fri Sep 06 19:17:49 EDT 2024 MediaSync: afterSyncData found 0 item(s) to remove
Fri Sep 06 19:17:49 EDT 2024 MediaSync: afterSyncData found 0 empty container item(s)
Fri Sep 06 19:17:49 EDT 2024 MediaSync: Exit afterSyncData
Fri Sep 06 19:17:49 EDT 2024 AppSyncWorker: Exit SUCCESS

 

I will note that I have a lot of videos downloaded, I travel a lot internationally so I sync a ton of media just out of habit (Probably ~200 gigs of media) so maybe I am hitting some sort of limit to the number of items that can be synced?  The Android phone is a Pixel Fold and I have ~100 gigs of space on the device.   If there is any other detail I can provide please let me know.  

Thank you!

[Luke 38863]

Quote

10) Renamed the files just in case a special character was messing it up

Hi, what did you rename?

Did the downloads complete?

[98DAM 0]

Oops I marked this as resolved by accident it is not resolved.  I renamed the files on my server to make sure they had no special characters and removed spaces just in case.  I changed it back after it did not seem to help.

I got a new phone last week and started fresh and everything seemed to download ok but I am back to having the same issues with it getting new episodes of shows it already downloaded.  It is different shows and the media has the same permissions as other files and they play fine with other media players and via emby without transcoding when I open them for streaming.

 

I'm this case I noticed the problem starting again when I got to over 200 gigs of media and similar to before if I look at the folder structure on the phone emby creates zero byte versions of the files

Edited September 30, 2024 by 98DAM

[Luke 38863]

What version of Emby server are you on now?

[98DAM 0]

I am currently on 4.9.0.30 Beta

[98DAM 0]

One new piece of data, the files that are stuck at ready to transfer show up as zero byte with permissions of 0770 while the working files have 0660.  Android doesn't let a user without root change the permissions on the emulated filesystem and my device is not rooted so I can't edit it on the fs directly to see if that helps.

I tried deleting the files but the transfer did not start.

Edited October 4, 2024 by 98DAM

[Luke 38863]

@98DAMcan you try removing all download jobs from the server, then remove all downloads from the device.

Then try to download something again. Are you able to download?

[98DAM 0]

Do you want me to remove downloads from all devices or just the devices with the issue?  I have an iPad and a couple of Android devices and it is the Android Devices which have the issues.  I don't mind deleting them all I just wanted to be sure before I wipe out my wifes device   

[98DAM 0]

I went ahead and deleted all the downloads from the devices and the server.  The issue came back almost immediately on the Android device on the same media that previously had the issue.  It does seem to be on specific media now but this same media did download ok previously but something caused it to decide this media is now bad...  The media is playable and I have the media quality set to original but the issue occurs if I set it to trancode as well.  

 

If I select just the problematic episodes to download they are stuck in the same state but the media plays fine from the device in Emby streaming

Edited October 22, 2024 by 98DAM

[98DAM 0]

One additional data point, the same downloads are working on another Android device and on my iPad.  It is almost like once the download fails and becomes stuck in ready to transfer it stays there.  Is it possible there is something either in the server of client queue that is messing this up?  

[98DAM 0]

I think I might have a theory of what is happening.  Yesterday I deleted all downloads on the device and the server plus I deleted the app and the entry for the device in Emby.  Once that was done all of the media downloads successfully including those that were an issue previously. I downloaded almost 300 gigs of media (non-transcoded) successfully. 

 

In my case Emby is not exposed to the internet and I VPN into my home network to access it, sometimes I am in locations where the VPN doesn't work or it drops frequently and it seems like when it is trying to download media sometimes it fails in a weird way that the server thinks the download is completed but the client thinks it hasn't (And I don't think it really has completed).   It only happens on my Android devices (It has happened over two different devices) and not iOS though maybe my usage model on the iPad is different?  The iPad is a cellular model and it also uses a VPN but maybe because it isn't with me all the time it doesn't get disconnected as much.

Is there anything I can provide to possibly help considering this new bit of information to narrow down the cause?

[Luke 38863]

@98DAMthat is very helpful. We'll try to simulate that in our testing. Thanks !

[98DAM 0]

I am not sure if it helps but I can consistently recreate the issue on Android now.  I currently have 3 TV Shows that have 1 or 2 episodes stuck on "Ready to Transfer".   This only seems to happen when I am traveling and not with every episode and I haven't been watching close enough to determine exactly what might be triggering it but I have a couple of theories:

1) I don't use a VPN frequently when traveling but I might trigger it periodically for whatever reason and then it gets disconnected which confuses the downloads.

2) Many of my trips are international and I cross through multiple countries in transit and I hit captive portals off and on which might be causing the issue of the client tried to connect and the captive portal gives it some sort of odd response.

3) It could be airplane wifi, particularly international flights.  The general instability of inflight wifi with drops and incredibly slow speeds at times if I start a VPN and it tries to sync I could see that causing issues for the client.

In the end it all points to just an unstable connection causing an issue. The Server thinks it is synched but the client is still waiting for it.  I don't see a way to clear this server-side but if there is a place in the database where that is tracked I am happy to try and edit that directly to see if it helps.  Is this tracked somewhere in the server that I can clear to work around it?  Deleting the downloads does not help and the same files won't download again unless I wipe everything, reset the client, and delete the record for the client on the server.

[randyg6403 0]

98DAM, you're not alone.  I have had the same issues since the beginning, many years,  downloading media when the internet is too slow to watch anything streaming,  while travelling internationally and on cruise ships.  So this isn't new, it's always been unreliable.   I'm quite sure you are correct about glitchy internet causing it.  Downloads at home, before you go, seem to work ok.

I have tried all the above workarounds, repeatedly.  Occasionally, a show will just download, without a problem.  8/10 times, it won't.   It can be stuck at "ready to transfer"  or "Transferring".

The Android client shows occasionally "download failed", but most often, nothing at all.    I think once you get a "failed" download, it doesn't want to try anymore, on any episode after.

My workaround is to point an HTTP file server at the transcode directory (HFS)  and use a resuming download manager(FDM) to get them.   The file names are obfuscated, so you need to guess them using the timestamps.  This works fine port forwarded, without the need for a VPN. Of course then I need to rename and transfer them to my android device with a usb cable on completion.   BUT,  they always complete, 100% success .   Lots of extra steps, but there isn't any other option that works for me.

(If you use this method, make sure you delete the unstarted downloads using the Emby admin options after manually transferring them, as I have found they can randomly start downloading at the most annoying times,  like a week later in the airport on the way home)

Interesting news I just read,  the windows Emby client, which was the worst at downloading before, has just been updated with all new downloading code.   It will still require a usb cable to get the media from notebook to my tablet to watch, but it's a step in the right direction, if it works now.   
(hopefully this new code supports some type of error correcting/resumable downloads, and perhaps some tiny amount of progress visibility past just "transferring" status)

see "Download Feature" https://emby.media/community/index.php?/blogs/entry/578-the-new-emby-app-for-windows-xbox/

 

[98DAM 0]

I am not sure if this helps at all but I saw the issue first hand this morning.  The client was downloading some new content and the Wi-Fi in the hotel dropped briefly causing the download to fail.   Later the notification stated that the download completed but the client shows "Ready to transfer" for that specific media.   Would it be helpful to grab any logs from this?

[Luke 38863]

On 1/6/2025 at 5:43 AM, 98DAM said:

I am not sure if this helps at all but I saw the issue first hand this morning.  The client was downloading some new content and the Wi-Fi in the hotel dropped briefly causing the download to fail.   Later the notification stated that the download completed but the client shows "Ready to transfer" for that specific media.   Would it be helpful to grab any logs from this?

HI, did the status later switch to downloaded?

[98DAM 0]

This is odd, all of the media now shows as Downloaded even media that had been stuck as "ready to transfer" for the last weeks.  I had 5 series with at least one episode (2 of them with multiple episodes) that were stuck at ready to transfer and somehow between when Monday and today they all seem to have resolved themselves despite being stuck in this state for weeks.  I did toggle the Download on Wifi only option while traveling to avoid battery drain so maybe that helped?  I had tried that earlier but I don't think I have ever left it in place for a few days...

I have a few trips the next 6 weeks so I can see if the issue returns but this is positive.

[PIkeoftheLake 8]

Yeah I'm struggling with this too.

Some media will download fine onto android devices, but others just won't download:

• When I look at the server, I will see it saying "Converting."
• I specified a temporary file path on the server and I can see it converting the file in the temp path.
• Once the file is done converting it will say, "Ready to transfer" on the server but nothing will happen on the android device.

[98DAM 0]

3 minutes ago, PIkeoftheLake said:

Yeah I'm struggling with this too.

Some media will download fine onto android devices, but others just won't download:

• When I look at the server, I will see it saying "Converting."
• I specified a temporary file path on the server and I can see it converting the file in the temp path.
• Once the file is done converting it will say, "Ready to transfer" on the server but nothing will happen on the android device.

Have you tried setting it to download in original quality to see if it is the issue is somewhere in the transcoding to copy pipeline?  I know you mention it transcodes successfully but when I started looking into this a lot of people reported that the conversion process caused them issues so I only use original quality now.

[PIkeoftheLake 8]

8 minutes ago, 98DAM said:

Have you tried setting it to download in original quality to see if it is the issue is somewhere in the transcoding to copy pipeline?  I know you mention it transcodes successfully but when I started looking into this a lot of people reported that the conversion process caused them issues so I only use original quality now.

Thanks.  I just tried downloading via original quality and still nothing.  I press download on the android device and nothing happens.

[98DAM 0]

Have you opened your Media Server to the internet or is it only accessible via your local network?

[PIkeoftheLake 8]

3 hours ago, 98DAM said:

Have you opened your Media Server to the internet or is it only accessible via your local network?

Both.  It is accessible outside my house and also accessible through local network.

I'm running my Emby server on a Truenas machine.

[98DAM 0]

Well there went that theory, I was having issues when I only had it available via VPN but it seems to have resolved itself after I went ahead and exposed it via a reverse proxy.  I did not want to do this but I was curious if the VPN was somehow causing downloads to get stuck.