Home networking woes: Uploading kills my connection

[jluce50 118]

Okay, so I've had a weird issue going on for a few weeks now. At random points throughout the day I'd notice my internet connection would slow to a crawl and latency would spike way up (along with significant packet loss). This would usually resolve after a little while, but sometimes I'd lose WAN connectivity completely (LAN was always fine). In either case a power cycle of the cable modem would fix it.

 

I've gone multiple rounds with Cox Business tech support and they could only ever detect a problem when my connection was completely dead. When I'm seeing the latency/packet loss they don't see anything wrong on their end. I bought a new SurfBoard 6141, which didn't fix the issue. I had a tech come out and, of course, they didn't see anything due to the intermittent nature.

 

So the other day it happened again and I just happened to see a notification on my phone that an app for syncing to Dropbox failed when I unplugged the modem. I got everything working again and restarted the backup job. Sure enough, my connection went to shit within a few seconds. With some more testing it turns out that any large upload brings my WAN connection to its knees. I've only got 15/3 so it's not hard to max out the upload bandwidth, but I never had this problem until recently.

 

After discovering that, I connected my modem directly to my PC and initiated a large upload. The ping times spiked, which may be normal, I'm not sure. I didn't have any packet loss though. I was hoping to determine if it was my router or the cable connection that was the problem, but I'm still not sure. I couldn't connect anything else so I couldn't precisely reproduce my earlier tests. The router is an ASUS RT-N66U running Shibby's Tomato. I've had the same configuration for quite a while and not had issues, but with the router in place I do get the packet loss and eventual dead connection. I'm not aware of anything usual about my router configuration other than a firewall script and an adblocking script. I did remove those and saw the same issue. I'm not aware of anything else that could be the culprit.

 

Almost everything points to the router except that when my connection was completely dead, Cox tech support was unable to "see" the modem. That part is really throwing me off.

 

I'm kind of at a loss at this point. Another service call will result in a charge if they don't find a problem on their end, so even though I can reproduce it I'm hesitant to go that route. Does anyone have any idea what might cause something like this?

Edited September 21, 2015 by jluce50

[ebr 14948]

I can't really help except to corroborate your experience - at least potentially.

 

Theoretically, speaking, let's say I have some program that downloads fairly large files from somewhere on a periodic basis.  This program happens to have a feature that allows me to limit the bandwidth it will consume which, as it turns out, is a very nice feature.  If I let this program use as much bandwidth as it wants, my connection to the internet will be completely crippled during any download operation.  If I limit it to, say, 3Mb/s then it does fine (just a little slower) and the internet for all the rest of the machines are fine too.

 

So, look in the settings for whatever it is that is doing this upload and see if it has the ability to limit the bandwidth it consumes.

[jluce50 118]

Yeah, I realize that maxing out the upload bandwidth would have adverse effects on other processes trying to upload. What doesn't seem normal is the packet loss and eventually losing the connection completely. Unfortunately, the app I mentioned in my post doesn't have the ability to limit bandwidth.

[d00zah 129]

Consider QOS via your router. Unfamiliar with Tomato, per se, but it ought to be quite capable of prioritizing various types of traffic. It may require some experimentation...

 

https://www.google.com/search?q=Shibby%27s+Tomato&ie=utf-8&oe=utf-8#q=shibby+tomato+qos

[jluce50 118]

I am familiar with QOS and Tomato has quite a good implementation. I was hoping to avoid that if at all possible, but it might be my best option. My biggest problem with it is that I think it's a band-aid and will just mask the real problem. I think I'm going to start by wiping the NVRAM on the router and seeing if that helps.

Edited September 21, 2015 by jluce50

[d00zah 129]

Fair enough. My thinking was that if you are considering throttling per app & that option was unavailable within that particular app...

Limiting upload to something < maximum would, at least, provide a data point & possibly preserve your connection?

 

Hope you find a resolution.

[jluce50 118]

Limiting upload to something < maximum would, at least, provide a data point & possibly preserve your connection?

 

Yeah, it may come to that yet. I went ahead and upgraded the firmware in addition to wiping the NVRAM and I'm testing now. So far I'm seeing the spike in ping times, but no packet loss, so that's encouraging.

[d00zah 129]

This seems to explain the issue well (different H/W, but mentions dropbox specifically)...

 

https://www.reddit.com/r/networking/comments/2sbd3a/packet_loss_when_maxing_out_upload_bandwidth_how/

 

although it may be that the updated F/W addresses the issue 'under the hood'. The problem(s) with these open-source router F/Ws include:

 

- Too many router variants to be supported (everyone has a preference)

- Always in an endless 'beta' cycle (I use DD-WRT)

Edited September 22, 2015 by d00zah

[jluce50 118]

Hmm, so it sounds like this is expected behavior then. I wonder why I didn't encounter this until recently? If I had to guess, Cox must have changed something with how they handle saturation on their end.

 

Turns out the firmware upgrade didn't fix anything after all. I went ahead and implemented QoS and that seems to be doing the trick so far. I just hate to limit my measly 3Mbps upload even further, but I guess I'll just have to deal with it. I'm planning on switching from my business plan to a 100/10 residential plan when my contract is up in a few months, so I'll just have to make do until then.

 

Thanks for your help!

Edited September 22, 2015 by jluce50

[d00zah 129]

Have you done a 'tracert' to the target server? Could be a circuitous route... oftimes beyond the ISP's control. I've had routing issues connecting to servers that have been routed cross-country & back again. Usually clears up over time, but can cause severe issues with VOIP traffic & such.

 

According to Cox DSLReports support forums, as well as this:

 

https://downdetector.com/status/cox-communications

 

you can see if others are experiencing similar issues.

 

Anyway, hope additional bandwidth helps, but if you continue to max out the connection...

Edited September 22, 2015 by d00zah

[Koleckai Silvestri 1150]

Yeah, I realize that maxing out the upload bandwidth would have adverse effects on other processes trying to upload. What doesn't seem normal is the packet loss and eventually losing the connection completely. Unfortunately, the app I mentioned in my post doesn't have the ability to limit bandwidth.

 

 

ISPs will search out and cripple home servers. If you're uploading a lot of data, you can be viewed as a home server providing information on the network.

[d00zah 129]

But with a current business account, hosting a server ought to be within the bounds of the TOS. & 3Mb isn't really a big bandwidth bite unless they are over-subscribed in that region.

 

& DownDetector DOES imply OKC area issues.

Edited September 22, 2015 by d00zah

[jluce50 118]

Have you done a 'tracert' to the target server? Could be a circuitous route... oftimes beyond the ISP's control. I've had routing issues connecting to servers that have been routed cross-country & back again. Usually clears up over time, but can cause severe issues with VOIP traffic & such.

 

I have run a tracert on a few different destinations and didn't see anything unsual. Uploading to any destination seems to trigger the issue.

 

 

ISPs will search out and cripple home servers. If you're uploading a lot of data, you can be viewed as a home server providing information on the network.

 

d00zah is correct, I am allowed to run a server under the business TOS.

Edited September 22, 2015 by jluce50

[d00zah 129]

Well, sounds like the band-aid may be desirable in lieu of losing the connection all together... until additional bandwidth is available, or area issues clear up. Good luck, jluce50!

[JeremyFr79 228]

Have you considered going with something more robust as far as a router goes? something like an older PC with a couple of NIC's running PFSense?  I had major issues with my old router running dd-wrt and getting to a point where it couldn't handle the number of open connections running.  I went out spent 50 bux on an old Dell 1950 Gen III threw PFSense on it, And I've never had an issue since.  PFSense is nearly as easy to use as DD-Wrt or Tomato but leaps and bounds above in capabilities and robustness.  You can literally run it an an Atom and have no issues.  My connection is 175/30 and it chews right through everything I throw at it unlike my old setup.

[jluce50 118]

Briefly, but I don't really think it's necessary at this point. The RT-N66U has been shown to be capable of 600Mbps on the LAN-WAN. 15/3 should be a non-issue. Once my speed increases I may reconsider, though.

[korvgryta 8]

The problem you are having could most likely be fixed by giving priority to the ACK packets. Since you have an assymetric connection, maxing out your upload will also limit your download. Prioritizing the ACK packets will greatly improve download speed even if you max out your upload.

 

This can be configured in some firewalls, not sure about the N66U.

 

Read more here: http://bandwidthcontroller.com/help/enterprise/advancedProcessingTcpAcknowledgementPrioritization.html

 

It can be done in DD-WRT and some other custom firwares or also directly on your machine. Can almost promise this will help you.

Edited September 24, 2015 by korvgryta

[legallink 187]

My experience with this issue previously ties it back to the router and not the connection.  Your dropping your WAN connection and having significant packet loss due to network saturation.  It isn't necessarily limited to just the amount you are pushing, but also the number of connections that are being made, or that your program is searching for.

 

I've solved this a few different ways: 1. It is possible that the router is starting to go bad.  I also use the N66 (although with Merlin firmware, not Tomato) and it holds pretty solid, although I have to do a router reboot every couple of months.  If you have any type of overhead introduced in the firmware, it will cause these types of issues when maxing out a connection. 2. QOS - as previously stipulated.  You are most likely overloading the routing table on your router.  I don't do much uploading nowadays, so I have limited recent experience.  3. Check to see if you have any firewall/security settings which when someone is trying to ping you it closes off the port.  Sometimes an AV, firewall, or network monitoring application can cause the same issues.

 

One point: it is unlikely they will find the issue on their side.  Possible, but unlikely.  When the connection goes dead, have you tried rebooting the Asus and not the modem?

[jluce50 118]

The problem you are having could most likely be fixed by giving priority to the ACK packets. Since you have an assymetric connection, maxing out your upload will also limit your download. Prioritizing the ACK packets will greatly improve download speed even if you max out your upload.

 

This can be configured in some firewalls, not sure about the N66U.

 

Read more here: http://bandwidthcontroller.com/help/enterprise/advancedProcessingTcpAcknowledgementPrioritization.html

 

It can be done in DD-WRT and some other custom firwares or also directly on your machine. Can almost promise this will help you.

 

Tomato can prioritize ACK packets, but it's under the QOS settings rather than the firewall. I don't know how it handles ACK when QOS isn't enabled.

 

 

My experience with this issue previously ties it back to the router and not the connection.  Your dropping your WAN connection and having significant packet loss due to network saturation.  It isn't necessarily limited to just the amount you are pushing, but also the number of connections that are being made, or that your program is searching for.

 

I've solved this a few different ways: 1. It is possible that the router is starting to go bad.  I also use the N66 (although with Merlin firmware, not Tomato) and it holds pretty solid, although I have to do a router reboot every couple of months.  If you have any type of overhead introduced in the firmware, it will cause these types of issues when maxing out a connection. 2. QOS - as previously stipulated.  You are most likely overloading the routing table on your router.  I don't do much uploading nowadays, so I have limited recent experience.  3. Check to see if you have any firewall/security settings which when someone is trying to ping you it closes off the port.  Sometimes an AV, firewall, or network monitoring application can cause the same issues.

 

One point: it is unlikely they will find the issue on their side.  Possible, but unlikely.  When the connection goes dead, have you tried rebooting the Asus and not the modem?

 

I was monitoring the connection count while reproducing the issue and I'm almost certain that the number of connections wasn't the issue. The program in question only had a handful opened.

 

JeremyFr79's post above actually got me thinking about using something like pfSense. I would just need to find a good dual-NIC NUC or Mini-ITX box capable of running VLANs for under $200 or so, which I haven't had much luck tracking down yet.