Jump to content

Downloads Vanishing?


Recommended Posts

Posted
On 12/28/2024 at 2:51 PM, CaptRodwell said:

Yes. I think it's happening on the client as I've seen it happen when the client is offline.

OK we'll try to chase this down. Thanks.

  • 3 weeks later...
Posted

I am having this same issue downloaded 5or6 episodes of a show at home it said they were downloaded when got to my destination opened up the app my downloads were no longer there🤷‍♂️ 

  • Thanks 1
  • 5 weeks later...
Posted

Same issue here. Updating to 4.9.0.38 did not make any difference.

client 3.4.36 - Android 15.

Sometimes downloads start, but then i get several episodes missing forever in "ready for download" state. Basically the same symptoms as other people described in this thread.

Posted
22 hours ago, Kigeki said:

Sometimes downloads start, but then i get several episodes missing forever in "ready for download" state. Basically the same symptoms as other people described in this thread.

"Ready for download" state is different from downloaded but disappearing after that. Is the Emby for Android app running and connected to the network ? What settings do you have for downloads on the Emby for Android app settings? When a download is requested, Emby Server would start convesion and then wait for the Emby app to download. 

Could we a look at specific example. You can enable logging on the Emby for Android app in app settings and then try to reproduce the problem where the download gets to the state of ready to download but not completing or starting the download and then get the emby server logs (embyserver and ffmpeg logs) and the Emby for Android logs and screenshots showing the download status on the server settings and the Emby for Android 

 

  • 4 weeks later...
Posted
On 2/24/2025 at 10:19 AM, sa2000 said:

"Ready for download" state is different from downloaded but disappearing after that. Is the Emby for Android app running and connected to the network ? What settings do you have for downloads on the Emby for Android app settings? When a download is requested, Emby Server would start convesion and then wait for the Emby app to download. 

Could we a look at specific example. You can enable logging on the Emby for Android app in app settings and then try to reproduce the problem where the download gets to the state of ready to download but not completing or starting the download and then get the emby server logs (embyserver and ffmpeg logs) and the Emby for Android logs and screenshots showing the download status on the server settings and the Emby for Android 

 

@Kigeki?

Posted
2 hours ago, Luke said:

Hi,

Sorry for the delayed response. I've just tried to reproduce the issue and everything worked surprisingly well. I've switched to 4.9.0.42-beta since then though (client is still on the same version). But last time, when I wrote my previous message, it was a frustrating experience. I was preparing for travel and wanted to download several series and it did not want to start, after the conversion had been done. The same behavior was observed on my wife's phone (apple). After poking around, deleting caches, deleting download queues, (and probably that is when I also switched to beta) I was able to download most of what I needed. Several episodes were still missing, but that was good enough.

I've just started several larger batches of downloads with conversions and will update here if it hangs.

 

  • Thanks 2
Posted (edited)

Unfortunately, I was able to reproduce it. Here is the details:
relevant android app logs:

Fri Mar 21 06:17:19 GMT+01:00 2025 AppSyncWorker: Start executing job.
Fri Mar 21 06:17:19 GMT+01:00 2025 CameraUploadWorker: Start executing job.
Fri Mar 21 06:17:19 GMT+01:00 2025 Found 0 server(s) for camera upload
Fri Mar 21 06:17:19 GMT+01:00 2025 CameraUploadWorker: Exit SUCCESS
Fri Mar 21 06:17:19 GMT+01:00 2025 ServerSync: Starting sync for server: 5456f9b6c26446df99444e9e7f629a74 'Caspar'
Fri Mar 21 06:17:19 GMT+01:00 2025 Getting response stream from http://192.168.200.33:8096/emby/system/info/public?format=json
Fri Mar 21 06:17:19 GMT+01:00 2025 Canceling requests with tag determineServerUrlTag-5456f9b6c26446df99444e9e7f629a74
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin checkLocalFileExistence
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin reportOfflineActions
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin getNewMedia
Fri Mar 21 06:17:19 GMT+01:00 2025 Getting response stream from http://192.168.200.33:8096/emby/Sync/Items/Ready?TargetId=0bd25342be9dda37
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: getReadySyncItems returned 2 items
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin getNewItem
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: getLibraryItem found existing item
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin getNewItem
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: getLibraryItem found existing item
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin syncData
Fri Mar 21 06:17:19 GMT+01:00 2025 Getting response stream from http://192.168.200.33:8096/emby/Sync/data
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Exit syncData
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin afterSyncData
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: afterSyncData found 0 item(s) to remove
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: afterSyncData found 0 empty container item(s)
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Exit afterSyncData
Fri Mar 21 06:17:19 GMT+01:00 2025 AppSyncWorker: Exit SUCCESS

server side logs for the same event:

2025-03-21 06:17:19.013 Info SyncService-0HNB7UN934IH3:00000003: http/1.1 POST http://‌‍‍192.168.200.33‌:8096/emby/Sync/data. Source Ip: ‌‍‍192.168.200.44‌, UserAgent: okhttp/4.12.0
2025-03-21 06:17:19.045 Info SyncService-0HNB7UN934IH3:00000003: http/1.1 Response 200 to ‌‍‍192.168.200.44‌. Time: 32ms. POST http://‌‍‍192.168.200.33‌:8096/emby/Sync/data

Out of 3 different tv-shows, only 2 random episodes from 2 shows are stuck.

It does not seem these logs highlight any obvious issues other than the fact that android app thinks that these 2 items already exist while they clearly don't `getLibraryItem found existing item`

Please see the attached archive for full logs (not sure why android logs start from 4am today though, I enabled them much earlier, but this morning's sync event is there at least).

Screenshots:

Spoiler

photo_2025-03-21_10-21-09.jpgScreenshot 2025-03-21 101106.pngScreenshot 2025-03-21 102344.png

 

debug.zip

Edited by Kigeki
Posted (edited)

Thanks for the logs and screenshots

I can see that for the two download jobs there were no transfer requests

TRIGUN STAMPEDE - S01E01
Download job #514
Converted at 2025-03-21 02:05:59.982

Arcane - S02E07
Download job #511
Converted at 2025-03-21 01:47:38.317

I would expect to see these requests from the android app 

POST /emby/Sync/JobItems/511/Transferred
POST /emby/Sync/JobItems/514/Transferred

after this request

POST /emby/Sync/data

As you indicated in the android app log - we have these log lines

Fri Mar 21 06:17:19 GMT+01:00 2025 Getting response stream from http://ip:8096/emby/Sync/Items/Ready?TargetId=0bd25342be9dda37
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: getReadySyncItems returned 2 items
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin getNewItem
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: getLibraryItem found existing item
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: Begin getNewItem
Fri Mar 21 06:17:19 GMT+01:00 2025 MediaSync: getLibraryItem found existing item

I am not sure if "found existing item" is referring to the Pixel 9 Pro already downloaded items

If it is reporoducible easily, would like to see a tcpdump of the requests and responses between the Pixel and the emby server. Are you able to do that and get me a zipped pcap file to go with new set of logs with debug logging enabled on the server and android and details of what did not transfer. Could you try the latest beta of the Emby for Android app? I see you are running on 3.4.36.  See this post for latest Emby for Android beta

Please send the zipped pcap and raw logs zip by private message

Edited by sa2000
Posted

@sa2000, It seems like the android app regularly retries syncing these 2 items so it should be easy to capture these sessions. I will setup a packet capture now and send the dump to you when it is ready.

  • Thanks 1
Posted (edited)

Thanks - and please enable debug logging on the server as well beforehand - in Server Settings > Logs > Settings and get me zipped copy of the raw logs folder with the android debug log and zipped pcap

Thank You

Edited by sa2000
  • Thanks 1
Posted

Thanks @Kigekifor the pcap and the logs. It looks to me like an Android app issue becuase the Emby Server has indicated to the app that there are two ready to transfer  

"Arcane (2021) - S02E07" and "TRIGUN STAMPEDE (2023) - S01E01"

These were returned in the 2 items list in the response to "GET /emby/Sync/Items/Ready"

and also later on when specifically requesting info about the download transfer jobs

Could you try the latest beta app 3.4.48 ?

Posted

I've installed 3.4.48 and cleared the cache but it did not fix the issue. Upon start the behavior is still the same with the same lines in the app log as in the previous snippets.

Posted
On 21/03/2025 at 23:47, Kigeki said:

I've installed 3.4.48 and cleared the cache but it did not fix the issue. Upon start the behavior is still the same with the same lines in the app log as in the previous snippets.

Thanks for trying. 

I presume the issue does not arise for alll season/series downloads - is that correct? Would you be able to reproduce the issue on a series or season that was not attempted to be downloaded before ? To see if we can get pcap and debug logs from both server and android app covering whole period from first start of downloads until the issue arises.

I will try to do some season downloads to see if it happens to me.

Was there a storage shortage issue at any time on the pixel during the downloads? The specific 2 files are not big 

Would you happen to have logs going back to when the first downloads were initiated for Arcane and TRIGUN STAMPEDE?

What mkv / ass files etc exist below the /transcode folder in paths like these

/transcode/65/514/
/transcode/64/511/

The 64 and 65 are download jobs identifiers and the 514 and 511 would be specific episode item job id's

 

Posted

Hey @sa2000,
Please see my answers inline:

Quote

I presume the issue does not arise for alll season/series downloads - is that correct?

Correct, it happens quite randomly. As you could see in one case only the first episode failed to download, and in the other - 7th out of total 9. Everything else, as well as other series I tried downloading finished successfully.

Quote

Would you be able to reproduce the issue on a series or season that was not attempted to be downloaded before ? To see if we can get pcap and debug logs from both server and android app covering whole period from first start of downloads until the issue arises.

I might try at some point, but cant promise. All this is very time consuming, sorry.

Quote

Was there a storage shortage issue at any time on the pixel during the downloads? The specific 2 files are not big 

Definitely not. More than 100GiB was available at all times. The fact that all other episodes downloaded successfully confirms this.

Quote

Would you happen to have logs going back to when the first downloads were initiated for Arcane and TRIGUN STAMPEDE?

Only the server logs without debug. I will dm you the full version. Short version here:

$ grep -E '/(510|511|512)/' emby/logs/embyserver-63878192764.txt | cut -c-250
2025-03-21 01:42:15.862 Info VideoEncoder: ProcessRun 'Encoding 11f1a6' Execute: /app/emby/bin/ffmpeg -loglevel +timing -y -print_graphs_file "/config/logs/ffmpeg-transcode-11f1a69b-630e-4e12-bee4-b27567e5e2f1_1graph.txt" -copyts -start_at_zero -init
2025-03-21 01:47:38.258 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /app/emby/bin/ffprobe -i file:"/transcode/64/510/11f1a69b-630e-4e12-bee4-b27567e5e2f1.mkv" -threads 0 -v info -print_format json -show_streams -show_format -show_data
2025-03-21 01:47:38.317 Info VideoEncoder: ProcessRun 'Encoding 814967' Execute: /app/emby/bin/ffmpeg -loglevel +timing -y -print_graphs_file "/config/logs/ffmpeg-transcode-81496766-fd46-444e-b6f5-365937e962c2_1graph.txt" -copyts -start_at_zero -init
2025-03-21 01:50:12.128 Info SyncService-0HNB7UN934IF0:00000003: http/1.1 POST http:///‌/‌‍‍192.168.200.33‌:8096/emby/Sync/JobItems/510/Transferred. Source Ip:  ‌ ‌‍‍192.168.200.44‌, UserAgent: okhttp/4.12.0
2025-03-21 01:50:12.141 Info SyncService-0HNB7UN934IF0:00000003: http/1.1 Response 204 to  ‌ ‌‍‍192.168.200.44‌. Time: 13ms. POST http:///‌/‌‍‍192.168.200.33‌:8096/emby/Sync/JobItems/510/Transferred
2025-03-21 01:53:31.914 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /app/emby/bin/ffprobe -i file:"/transcode/64/511/81496766-fd46-444e-b6f5-365937e962c2.mkv" -threads 0 -v info -print_format json -show_streams -show_format -show_data
2025-03-21 01:53:31.959 Info VideoEncoder: ProcessRun 'Encoding 1e56f9' Execute: /app/emby/bin/ffmpeg -loglevel +timing -y -print_graphs_file "/config/logs/ffmpeg-transcode-1e56f994-b29a-40bd-9b8d-c2aac21dedd8_1graph.txt" -copyts -start_at_zero -init
2025-03-21 01:59:09.508 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /app/emby/bin/ffprobe -i file:"/transcode/64/512/1e56f994-b29a-40bd-9b8d-c2aac21dedd8.mkv" -threads 0 -v info -print_format json -show_streams -show_format -show_data
2025-03-21 01:59:14.789 Info SyncService-0HNB7UN934IF7:00000003: http/1.1 POST http:///‌/‌‍‍192.168.200.33‌:8096/emby/Sync/JobItems/512/Transferred. Source Ip:  ‌ ‌‍‍192.168.200.44‌, UserAgent: okhttp/4.12.0
2025-03-21 01:59:14.796 Info SyncService-0HNB7UN934IF7:00000003: http/1.1 Response 204 to  ‌ ‌‍‍192.168.200.44‌. Time: 8ms. POST http:///‌/‌‍‍192.168.200.33‌:8096/emby/Sync/JobItems/512/Transferred

As you can see 510 finished transcoding and transferred successfully as well as 512. While 511 finished transcoding and was never downloaded.

Quote

What mkv / ass files etc exist below the /transcode folder in paths like these

Everything is there as expected. I've also downloaded these files to my PC and they seem fine, I can play the video without issues.

       17 Mar 21 01:07 /storage/emby_transcoding/64
       54 Mar 21 00:47 /storage/emby_transcoding/64/511
291966442 Mar 21 00:53 /storage/emby_transcoding/64/511/81496766-fd46-444e-b6f5-365937e962c2.mkv
       17 Mar 21 01:19 /storage/emby_transcoding/65
       98 Mar 21 01:07 /storage/emby_transcoding/65/514
258266592 Mar 21 01:07 /storage/emby_transcoding/65/514/344974f4-c4b9-4e55-b748-d1ea60cdb3ed.mkv
    52944 Apr  5  2024 /storage/emby_transcoding/65/514/b01a0a3a66ac46f58c30a23c033c12c4.ass

 

  • Thanks 1
Posted

@Kigeki In Emby App Settings for Downloads on the Pixel, did you make changes to the target location from the default of Internal Storage ?  Has it always been the same target location?

I have been doing repeated downloads and so far I have not been able to get the issue.

 

Posted
15 hours ago, Kigeki said:

I will dm you the full version.

OK - will wait for those logs plus all the android app logs that may exist.

Posted
55 minutes ago, sa2000 said:

@Kigeki In Emby App Settings for Downloads on the Pixel, did you make changes to the target location from the default of Internal Storage ?  Has it always been the same target location?

I played with this setting before, around the time of my initial post, but it did not help and I reverted to "internal storage". All the latest tests were conducted with "internal storage" selected and I did not touch any other download settings.

Quote

I have been doing repeated downloads and so far I have not been able to get the issue.

I’d like to point out that during this latest test, I initiated the download of 3 different TV series, totaling 4 seasons with approximately 10 episodes each—around 40 episodes in total. Transcoding was enabled, and the entire process took about 4 hours to complete (start: 2025-03-20 22:42:00.342, end: 2025-03-21 02:17:58.605). Only 2 episodes got stuck for me, though, so it might just be that you got lucky 😃

Taking a blind guess: could it be that Android kills the app (for whatever reason) in the middle of the process, while the app marks episodes as "done" in its internal database before they're actually completed?

Posted
3 minutes ago, Kigeki said:

I played with this setting before, around the time of my initial post, but it did not help and I reverted to "internal storage".

I was just thinking if the target location changed during the downloads 

Are you able to see if there are any downloads on the external storage ?

Posted (edited)
38 minutes ago, sa2000 said:

I was just thinking if the target location changed during the downloads 

Are you able to see if there are any downloads on the external storage ?

I'm certain it didn't change. I've just double checked the external Downloads folder to confirm there is no Emby downloads.

2 hours ago, sa2000 said:

plus all the android app logs that may exist.

They don't, unfortunately, I enabled them the next morning when I was instructed to do so.

Edited by Kigeki
  • Thanks 1
Posted

@KigekiThanks for all the diagnostics provided. I have not been able to reproduce the issue trying out multiple series downloads. I have now referred the issue with the diagnostics to the development team

 

  • Thanks 1
Posted (edited)

@KigekiIn my attempt to reproduce the issue, I did 5 TV shows downloads and 3 of them completed ok but 2 were in state of ready to transfer and transferring. I have 50 episodes flagged by the server as ready to transfer but the android app was logging the same log lines as you had in your logs for these 50 episodes

I discovered whilst trying to share my android logs through the app that the device was short on storage - i had a pop up

So I deleted the download job for the 3 series that completed ok and after that the 50 ready to transfer got redued to 38

So could you please when the problem arises, go to settings on the device and find out how much free space there is at that time on internal storage for the device

 

 

 

 

Edited by sa2000
  • Thanks 1
Posted (edited)

@Kigeki An update - after the earlier shortage of storage and then freeing up storage - download transfers resumed but i have ended up now with same situation as your case - with 2 episodes not being downloaded - with over 15Gb free internal storage available. I do not know at this stage, if the earlier transient storage shortage was a factor in leading to the current state

Edited by sa2000
  • Thanks 1
Posted

@sa2000I repeated the test overnight. I cleared all download queues, deleted all downloaded episodes from the phone, and added 5 different series to the download queue again.

Here is the current state as of this morning:

  • 3 series downloaded completely without issues
  • 1 episode is stuck in 'ready for download' status for Trigun (interestingly, it's S01E01 again)
  • 7 episodes are stuck in 'ready for download' status for Arcane (S02E01-07)

There is still plenty of space available.

Screenshots:

Spoiler

image.thumb.png.dbf5a711d80079de59dadf1132e5d258.pngScreenshot2025-03-25112148.png.9d859da0c5ca29ed410b4e5d7a4d50cf.pngScreenshot2025-03-25112203.png.a431ada7382e72861f0fde3a30766ddb.pngScreenshot2025-03-25112225.png.7f16a6ac528122da06e5d4f3a8ef9e4c.png

 

  • Thanks 1
  • 2 weeks later...
Posted

@KigekiWe have fixed a problem relating to downloads in beta 4.9.0.43 which is for cases when there are loads of episodes being downloaded. I would like to know if with this change, the problem still occurs.  Could you repeat the test for series/seasons that have not been downloaded before and if it still arises, please let me get fresh set of debug log files - from both android and the server and screenshots and info on when the downloads were initiated and which episodes remained in ready to transfer state

Thanks

  • Like 1
Posted

@sa2000The issue is still reproducible on beta 4.9.0.43. I'll send you the logs in DM shortly.

  • Thanks 1

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...