Dropbox Sync/Upload Very Slow

[M4TTD9TX 5]

Hi,

Firstly thanks for getting the Dropbox plugin working again with the API v2, however, I've noticed the sync/transfer to Dropbox is incredibly slow now. Sometimes the task is running for days and it only manages to eventually upload one 800mb file. I have a relatively fast upload connection and any other upload/sync to Dropbox (via their website or app) is at least 3-4 MB/s.

 

I understand that Dropbox do control the transfer speed to approx 75% of ones upload speed (can be disabled), but monitoring the server upload speed from the Emby Dropbox plugin shows it hardly gets to 1 MB/s.

 

Needless to say it wasn't as slow as this prior to the API update, so wondering whether there's something I'm missing? I've gone through the Emby/Dropbox settings, all to no avail.

 

Many thanks,

 

Server log added for completeness:

Emby Version 3.2.60.6

Dropbox Plugin 3.1.5.0

 

server-63650361600.txt

[Luke 37180]

Hi, what kind of storage is this on? it could be related to that.

[M4TTD9TX 5]

Hi Luke,

Our Emby Server/Win 10 OS is on a dedicated Samsung 960 Pro SSD, with the media spanned across 10x 6TB SATAIII (60TB) drives in the same system.
Transfer rates across the drives are fast, as is across our gigabit hardwired/wireless network. Any streamed media (direct from me) goes out capped at 3MB/s per user (not sure what they'd see from Dropbox??).

 

Was hoping for some sort of backend Dropbox API configuration command that's holding the transfer back to Dropbox (business uncapped storage), as before the API v2 was introduced (thanks again), it was transferring in a timely manner.

 

It's been running for another 12 hours last night and transferred nothing by the looks of it so far :-(

 

 

[Luke 37180]

It's really strange because all we really did was update the authentication for the api changes.

 

@@randomevents did you observe this behavior while updating the plugin to the new api? Thanks !

[randomevents 38]

With a fresh server updated to 3.60.7 and using plugin version 3.1.5, I'm seeing just north of 3MB/s (on a 300Mb/s connection), log is showing about a 10MB chunk every 3 to 4 seconds. Which is about the same if I drop a file into the browser.

 

 

 

2018-01-02 21:59:09.434 Info HttpClient: POST: https://content.dropboxapi.com/2/files/upload_session/append_v2
2018-01-02 21:59:12.224 Debug Dropbox: Received HttpResponseInfo
2018-01-02 21:59:12.293 Debug Dropbox: Dropbox-API-Arg: {"cursor": {"session_id": "AAAAAAAAAEPoOrMUnauaDA","offset":1310720000},"close": false}
2018-01-02 21:59:12.293 Debug Dropbox: Content: Binary (10485760) bytes
2018-01-02 21:59:12.293 Debug Dropbox: Send httpRequest
2018-01-02 21:59:12.293 Info HttpClient: POST: https://content.dropboxapi.com/2/files/upload_session/append_v2
2018-01-02 21:59:16.299 Debug Dropbox: Received HttpResponseInfo
2018-01-02 21:59:16.385 Debug Dropbox: Dropbox-API-Arg: {"cursor": {"session_id": "AAAAAAAAAEPoOrMUnauaDA","offset":1321205760},"close": false}
2018-01-02 21:59:16.385 Debug Dropbox: Content: Binary (10485760) bytes
2018-01-02 21:59:16.385 Debug Dropbox: Send httpRequest
2018-01-02 21:59:16.385 Info HttpClient: POST: https://content.dropboxapi.com/2/files/upload_session/append_v2

 

I'd like to look at the logs but for some reason I can't download anything (permissions?) @@Luke?

Edited January 3, 2018 by randomevents

[Luke 37180]

@@Abobader can you give him permission? thanks.

[M4TTD9TX 5]

Thanks for looking in to this, it's really appreciated.

 

For some reason dragging and dropping a file into a Dropbox web browser sees faster upload speeds than when Emby tries to sync it (same file). Or indeed via the desktop/folder app.

For elimination, I am moving ALL the transcoded files waiting to sync, over to the O/S SSD drive (and changing sync path) to see what that does to the sync speed.

I've checked ports and firewalls etc, even the ISP, but because I get different speeds between raw uploading and the Emby sync, my pointing finger keeps landing at the plugin. I've tried inputting huge numbers to the Upload speed limit box, or nothing, and ticked/unticked the Enable full speed conversion etc

 

This is a great feature and I thank you for it being there, but not sure if anyone else is struggling with the sync speed out there (conversion is rapid).

 

Thanks again

[Abobader 2953]

@@Abobader can you give him permission? thanks.

 

Good day,

 

Done.

 

My best

[randomevents 38]

@@Abobader thanks. was something changed recently? 

 

@ Okay, I'm looking at your logs and I'm trying to decipher what's going on. As you mentioned it is slow and I can't narrow it down as to why. However, your logs are littered with syncing  and port map errors and I'm wondering if the latter is the problem.

 

2018-01-01 00:06:29.065 Error PortMapper: Error creating port map: InternalServerError

2018-01-01 00:06:31.677 Debug PortMapper: Found NAT device: uuid:Upnp-BasicDevice-1_0-BC664116522D::urn:schemas-upnp-org:service:WANIPConnection:1

2018-01-01 00:06:31.677 Debug PortMapper: Calling Nat.Handle on uuid:Upnp-BasicDevice-1_0-BC664116522D::urn:schemas-upnp-org:service:WANIPConnection:1

 

You can see here, that it posts just fine, but there's a big delay on the response from Dropbox.

2018-01-01 14:07:38.699 Info HttpClient: POST: https://content.dropboxapi.com/2/files/upload_session/append_v2

2018-01-01 14:07:47.639 Debug Dropbox: Received HttpResponseInfo

 

 

[Abobader 2953]

 

@@Abobader thanks. was something changed recently? 

 

 

 

 

Good day,

 

Yes, since dev's group suggest not everyone could see the full log file that upload by attachment, so selected personals and groups now will allowed to view the attachment.

 

Sorry for all this, and thanks for your understanding the matters.

 

More info in this thread:

 

https://emby.media/community/index.php?/topic/53986-compromised-login-investigate-odd-ips/page-4&do=findComment&comment=526673

 

And the posts after that, Angelblue05 explain it well.

 

My best 

[randomevents 38]

Good day,

 

Yes, since dev's group suggest not everyone could see the full log file that upload by attachment, so selected personals and groups now will allowed to view the attachment.

 

Sorry for all this, and thanks for your understanding the matters.

 

More info in this thread:

 

https://emby.media/community/index.php?/topic/53986-compromised-login-investigate-odd-ips/page-4&do=findComment&comment=526673

 

And the posts after that, Angelblue05 explain it well.

 

My best 

 

Thanks for the explanation, now I can get back the hell that is debugging some of Emby Theater issues.

[Abobader 2953]

Well done!

[M4TTD9TX 5]

 

@@Abobader thanks. was something changed recently? 

 

@ Okay, I'm looking at your logs and I'm trying to decipher what's going on. As you mentioned it is slow and I can't narrow it down as to why. However, your logs are littered with syncing  and port map errors and I'm wondering if the latter is the problem.

 

 

 

You can see here, that it posts just fine, but there's a big delay on the response from Dropbox.

 

 

 

@@randomevents Thanks for taking a look! I have previously opened up Dropbox router forwarding ports, but they seem to be more for LAN sync and website uploading, however, I'll interrogate the UPnP port mapping side of the system in case the plugin/Emby is trying to access ports not opening automatically for them.

 

On with the good fight, thanks guys!