MacInTouch Amazon link...
Channels
Other, Products, Internal/development, Questions
How do you set this up at a router level?
The Netgear router app (called "Nighthawk") has a built-in Ookla Speedtest module, which measures speeds at the router, not between the router and devices. (Documentation says: "detects the download and upload speed from your router to the internet.") Clever idea, really. Just wish it had an "automation" function, so I didn't have to check manually every time.

I don't need huge precision. Just trying to capture a record of data to judge overall compliance by the ISP with its contract.
 


And it should be noted that you almost certainly don't have any guarantee - either verbally or in writing...
If you want an actual guarantee...
... There is almost no good reason why a small business or an individual needs that level of service.
Yes, I'm well aware that neither do I have a "guarantee" nor do I need that level of service. I'm merely trying to collect data, by automating the testing function, with which to measure the ISP's actual performance against their sales pitch. Don't need a "guarantee" for that, any more than Consumer Reports does when it measures automobile performance against claims. Public pressure will do the job, especially when local governments approach renewal time for franchise agreements.

And, to be fair, maybe I'll discover that the ISP really does pretty much meet its claims! My anecdotal observations so far don't support that, but I'm certainly open-minded. I've begun keeping a manual log. If the electric power company matched the ISP's level of performance so far, the government would rake them over the coals.
 



Ric Ford

MacInTouch
Question: Does anyone have experience with automating this process of checking ISP speeds? Is there software available (low-cost or free, because it's not for business or WAN use) which...
  • monitors download/upload speeds
  • can be deployed to measure at the router level
  • can retain-and-make-available spreadsheet-style data over time
That's a pretty tall order, though someone could probably program a command-line utility for the Mac pretty easily to do something like this. Alternatively:
TestMy.net said:
Automatic Speed Test
Want to automate your speed test and monitor your connection over time? This tool will automatically retest your Internet connection on a set interval and log the test results for later retrieval. The Automatic Speed Test can provide data that may aid in Internet troubleshooting. Simply set the test interval, click start and forget about it. Then, come back later and retrieve your results. TMN requires only your web browser, has trusted test results and is always free.
You might also want to look into "SNMP" and related software tools (if your router supports SNMP). Here are a few:
 


It is probably obvious to the readers, but here is an experience I had:

I had a case where I was sure that something was wrong with my download speeds, which Ookla indicated. As it turned out, Google was uploaded a backup without notifying me and without my permission. When an upload saturates the upload capacity of the internet service, the download speed grinds to a stop, simply because the upload connection is used to fetch a download and to check the upload.

It was not obvious, because the Google uploads did not show up my download/upload indicator in the menu bar. So, issues with the upload can dramatically impact download speeds, and the upload channel can be easily saturated.
 


Absolutely, Tim! I understand the strain, and I respect and appreciate your kind comment.
The issues I'm tracking predate the pandemic and are chronic in my (rural) area. Of course, I don't expect any ISP to meet 100% of its contract, and I'm very respectful of the extraordinary demand now being experienced. But electric power utilities are held to a much higher standard - even in high-demand times, and with the now-essential nature of broadband access, ISP need to be held to a higher standard, too. I fully understand the pipes are not big enough. That's one of the key issues local governments and ISPs need to address. In my area, physical (coax) cables now supplying internet to residences haven't been replaced or upgraded since they were installed for cable TV in the 1980s. It's time for some investment. The data I'm seeking is merely a starting point for that effort.
Ralph, I completely understand, and if I were having chronic speed or connection issues, I'd definitely push the ISP. If you're running speed tests during non-peak hours, you should expect to achieve close to, if not right on, your speed package.

There are two ways to look at results from running speed tests - first, a test to a very close server (typically run by your ISP) should yield the subscribed speeds on off-peak hours. A second test to other "remote" speed test servers is going to measure packet access across the ISP's core network. A traceroute to that speed test server should show the path through an ISP's core network. In a traceroute, high intermediate ping times are not generally an issue, as those intermediate routers consider ICMP traffic [pings] very low priority.

As for ISPs being on the same level as electric utilities - for the most part, I would completely agree. And, speaking only for the ISP I work for, we do our very best to increase capacity as needed, including hosting various cache and content servers from Netflix, Google, Akamai, Facebook, etc., as well as pushing more bits through larger pipes. And, of course, increasing interconnect pipe sizes with the outside world at various points on the edge of our network.

Good luck - and after the current crisis has passed, gently nudge your ISP (and get neighbors to do as well). Hopefully they'll listen - but, I do read some really horrendous experiences with various (unnamed) carriers.
 


You have to be careful about making "blanket" assumptions based on speed tests, because there are so many variables.

For example, when we turned up a 100-meg fiber connection to a customer, he was going bonkers running speed tests, because he could only get a fraction of his pipe. We discovered that he was basing his speed test on his antique 802.11b laptop. When he connected via Ethernet, he saw his full speed capacity. ISPs generally can't control the customer equipment.

We see similar situations when the customer complains about "not getting what I'm paying for". Upon inspection of what's going on with the customer connection, we often find they are connecting wirelessly with a bunch of local frequency interference, or a low signal far away from the wireless access point.

Most recently, with everybody working remotely and the kids home, there is a lot of contention for bandwidth between VPN, gaming consoles and HD/4K video streaming all getting choked locally.

A little customer education goes a long ways. An accurate speed test would be with only one device with an Ethernet connection connected directly to the ISP interface. Even then, you need to pick a "local" speed test site that's within the control of your provider. If you're in Seattle and using a speed test site in Guam, your "results" won't be accurate.
 


Yes, I'm well aware that neither do I have a "guarantee" nor do I need that level of service. I'm merely trying to collect data, by automating the testing function, with which to measure the ISP's actual performance against their sales pitch.
But with any residential service, oversubscription means your results will depend greatly on the usage of your neighbors. This is especially true on a cable infrastructure, where your neighbors share the same physical cable, but it can also be true on fiber-to-the-home installations, because the physical fiber is still shared. Even if every customer has a dedicated wavelength to avoid conflicts on the fiber, they will still come together at a common access router, which will almost certainly be oversubscribing its link to the rest of the provider's network.

If you run your tests when the network is mostly idle (e.g. middle of the night), you should see the full speed you paid for. If you run it when the network is busy - like in the evening when lots of people are streaming movies or during the day (at least right now) when lots of people are working from home, you may not be able to see those maxima.

And, of course, as others have pointed out, your LAN connection to the router and other traffic on your LAN can significantly impact speed test results.

In other words, what you talk about sounds simple, but it really isn't under real-world conditions.
 


You have to be careful about making "blanket" assumptions based on speed tests, because there are so many variables.
For example, when we turned up a 100-meg fiber connection to a customer, he was going bonkers running speed tests, because he could only get a fraction of his pipe. We discovered that he was basing his speed test on his antique 802.11b laptop. When he connected via Ethernet, he saw his full speed capacity. ISPs generally can't control the customer equipment.

We see similar situations when the customer complains about "not getting what I'm paying for". Upon inspection of what's going on with the customer connection, we often find they are connecting wirelessly with a bunch of local frequency interference, or a low signal far away from the wireless access point.
We see this with the fiber (or HFC) circuits we sell to business customers, and we have a very specific SLA with them. If they feel a circuit isn't running at full subscribed speed, we have testers that can prove out the circuit and provide them the data.

Again, I work for a small ISP, so when circuit speed issues come up, they land in the department I work for, to investigate and assist in resolving. I appreciate the thoughtful comments.
 


You have to be careful about making "blanket" assumptions based on speed tests, because there are so many variables. For example, when we turned up a 100-meg fiber connection to a customer, he was going bonkers running speed tests, because he could only get a fraction of his pipe. We discovered that he was basing his speed test on his antique 802.11b laptop. When he connected via Ethernet, he saw his full speed capacity. ISPs generally can't control the customer equipment.

We see similar situations when the customer complains about "not getting what I'm paying for". Upon inspection of what's going on with the customer connection, we often find they are connecting wirelessly with a bunch of local frequency interference, or a low signal far away from the wireless access point.

Most recently, with everybody working remotely and the kids home, there is a lot of contention for bandwidth between VPN, gaming consoles and HD/4K video streaming all getting choked locally.

A little customer education goes a long ways. An accurate speed test would be with only one device with an Ethernet connection connected directly to the ISP interface. Even then, you need to pick a "local" speed test site that's within the control of your provider. If you're in Seattle and using a speed test site in Guam, your "results" won't be accurate.
When our service got updated to 200 Mbps, I was unable to see the speed increase after upgrading my modem to DOCSIS 3.1. Imagine my chagrin after many tests and phone calls to realize that the cable between the modem and my router was only Cat 5. It probably was vestigial from when I initially switched from dial-up to cable internet many years ago.
 



You folks are really wonderful! Thank you all for this rich and thoughtful discussion. You're helping me be careful and thoughtful and cautious. We'll see where this leads...
Thanks!
 



When our service got updated to 200 Mbps, I was unable to see the speed increase after upgrading my modem to DOCSIS 3.1. Imagine my chagrin after many tests and phone calls to realize that the cable between the modem and my router was only Cat 5. It probably was vestigial from when I initially switched from dial-up to cable internet many years ago.
Cat 5 wire shouldn't make a difference. Over short distances, Cat 5 can handle gigabit speeds with no problems. Even over longer distances (up to 100 meters), Cat 5 will work if the cables and patch panels are all up to spec. (Cat 5e is recommended, because you don't have to be nearly as precise to get a clean signal, but Cat 5 will work at gigabit speeds, especially if the cable lengths are not approaching the maximum.)

It may be, however, that your router's WAN port only supports 100Mbps Ethernet and not Gigabit. But that's probably not the case, because you implied (without actually stating) that swapping the cable solved the problem.

It may also be that your old Ethernet cable didn't wire all four pairs. 10Mbps and 100Mbps Ethernet only use two of the four twisted pairs, and some cheap cables meant for those speeds save a few cents by only having four wires (two pairs) instead of the usual eight (four pairs). If you look through the clear plastic connector, you should be able to see if there are four or eight wires going to the pins in the connector.

Finally, it may be that you've got a "crossover" cable. For 10M and 100M connections, one pair is used for transmit and the other is receive. When connecting a host to a switch/hub, one device is wired opposite to the other (MDI vs. MDI-X) in order to allow cables to be wired straight-through. In the past, when connecting a host to a host or a switch to a switch, you would need a "crossover" cable where the transmit and receive pairs are swapped. Or (for switch-to-switch connections), you would use a dedicated "uplink" port that is wired as a host. More modern equipment can automatically set each port to MDI or MDI-X mode in order to eliminate the need for crossover cables.

Gigabit Ethernet does not require crossover cables. All four pairs are used for both transmit and receive. If a "half crossed" cable (where only wire pairs 2 and 3 - those used by 10M and 100M Ethernet - are crossed), the link will either auto-negotiate to 100M or won't come up at all. Gigabit Ethernet should support "fully crossed" crossover cables, where pairs 2 and 3 are crossed and pairs 1 and 4 are crossed, but those are not commonly found and are really never necessary.

If you look through the plastic connectors on your cable, you should be able to see if the cable is crossover or not, and how the wires are crossed. A straight cable will have the colors appearing in the same order on both connectors. A crossover cable will have the colors appear in a different order on each end. (If your cable's connectors are a kind that doesn't let you see the individual wires, you can also check this with a cable tester or a continuity tester.)
 


DFG

But with any residential service, oversubscription means your results will depend greatly on the usage of your neighbors. This is especially true on a cable infrastructure, where your neighbors share the same physical cable, ...
Well, not always true.

I have been collecting data on my internet connection (Comcast Xfinity cable, located in San Jose, CA) for years now, using a cron job on my Mac, which is always on, to run scripts measuring ping times (to google) and download speed (with speedtest.cli) every 30 minutes. cron is good, solid UNIX technology, never fails, and much easier to setup than launchd...but I digress.

My service tier, "Performance Plus Internet," is nominally 75 Mbps down, but Comcast has been slowly increasing the speed (the upload speed, however, remains at a paltry 6 Mbps). Incidentally, this tier is no longer available for new customers, who get "up to 100 Mbps."

I was curious to see how the ISPs are holding up to everybody working from home, which in my area started on March 14.

Here's a plot of the internet speed, and here are the max. ping times. From about 9 AM to 11 PM the likelihood of getting a longer-than-averge ping increases.

I was surprised to see that the service is holding up great. There are no easily discernible daily usage patterns (though one could argue that the early hours of the day have fewer dropouts). The median speed is 88 Mbps, and average max. ping time 12 ms. Keep in mind that I run this while the rest of the household is also using the internet, that could explain some of the speed drops.
 



I'm not sure that is correct. There are now lots of GigE PoE switches that achieve 1G speeds but have two of the pairs used for power....
For most standard PoE implementations, power is carried over the same wires as data, so all four pairs are still used for gigabit and faster speeds.
Wikipedia said:
Power over Ethernet
...
Standard Implementation
A phantom power technique is used to allow the powered pairs to also carry data. This permits its use not only with 10BASE-T and 100BASE-TX, which use only two of the four pairs in the cable, but also with 1000BASE-T (gigabit Ethernet), 2.5GBASE-T, 5GBASE-T, and 10GBASE-T which use all four pairs for data transmission. This is possible because all versions of Ethernet over twisted pair cable specify differential data transmission over each pair with transformer coupling; the DC supply and load connections can be made to the transformer center-taps at each end. Each pair thus operates in common mode as one side of the DC supply, so two pairs are required to complete the circuit. The polarity of the DC supply may be inverted by crossover cables; the powered device must operate with either pair: spare pairs 4–5 and 7–8 or data pairs 1–2 and 3–6. Polarity is defined by the standards on data pairs, and ambiguously implemented for spare pairs, with the use of a diode bridge.
 


... I was curious to see how the ISPs are holding up to everybody working from home, which in my area started on March 14.
Here's a plot of the internet speed, and here are the max. ping times.
I was surprised to see that the service is holding up great. There are no easily discernible daily usage patterns (though one could argue that the early hours of the day have fewer dropouts). The median speed is 88 Mbps, and average max. ping time 12 ms. Keep in mind that I run this while the rest of the household is also using the internet, that could explain some of the speed drops.
As I've noted previously, I work for an ISP, and we've definitely seen very different traffic patterns. Obviously our overall usage has increased, but we're weathering well, all things considered. We've had to escalate some planned upgrades, and even Netflix has upgraded feeds to a number of their servers in our network to handle the increased demand.

Stay safe everyone.
 


As I've noted previously, I work for an ISP, and we've definitely seen very different traffic patterns. Obviously our overall usage has increased, but we're weathering well, all things considered. We've had to escalate some planned upgrades, and even Netflix has upgraded feeds to a number of their servers in our network to handle the increased demand.
Stay safe everyone.
Only symptom I’ve noticed is outgoing mail regularly spends a few seconds in the Out box while, before, I rarely evev saw the Out box. Browsing and downloads seem normal.
 


As a recent upgrader to truly high speed, I find things, after two days so far, a very pleasant change for the most part. I can now manage large files from home.

One thing I do notice, that is a bit odd, is web navigational lag-time. Don't get me wrong, the basics of high-speed, and its utility for family and homeuse, is rather awesome... but the lag-time in navigating/browing, loading new pages, etc. is a bit of a mystery... we got to the front door in a nano-second, but... then... it to...oook... 9 secondssss to ohhhpennn the doooooooor.

Another for instance, when previewing this post... lots of purple progress bar action at the top of this page while preview is loading.
 


Ric Ford

MacInTouch
Another for instance, when previewing this post... lots of purple progress bar action at the top of this page while preview is loading.
John, a quick speed test of this page I just did rated it 99/100 with "First Contentful Paint" in 0.7 sec.

Now, if you're not logged in, you may get some delays from the Amazon search box. And, rarely, there might be a slowdown on our hosting server, but MacInTouch is just about the last site in the world you can complain about being slow!

Our COVID page is even faster:
I'm not sure what caused your delay – perhaps a different website doing something behind the scenes in a different tab, or something else on your computer (iCloud, Dropbox, etc.), but I'm pretty sure the delay wasn't due to MacInTouch.

If you want to see a really bad/slow website, try loading The Huffington Post, for example and the piles and piles of s**t they load into and out of your computer when you go there (take a look at the waterfall chart and all that's going on behind the scenes):
WebPageTest.org said:
Compare, for instance:
WebPageTest.org said:
You should also be sure you're using a good DNS server, as a poor choice there could slow things down. See, for example:
 


I'm not sure if it has already been reported here, but in updating to the latest iOS app for Consumer Cellular this morning, I learned that they have replaced their high-end 25GB data plan with an "unlimited" plan; their website confirms it.
 


One thing I do notice, that is a bit odd, is web navigational lag-time. Don't get me wrong, the basics of high-speed, and its utility for family and homeuse, is rather awesome... but the lag-time in navigating/browing, loading new pages, etc. is a bit of a mystery... we got to the front door in a nano-second, but... then... it to...oook... 9 secondssss to ohhhpennn the doooooooor.
If you’re consistently getting stalling when clicking on new links, then that is most likely a DNS server issue.
 


I'm not sure if it has already been reported here, but in updating to the latest iOS app for Consumer Cellular this morning, I learned that they have replaced their high-end 25GB data plan with an "unlimited" plan; their website confirms it.
A very good plan, but it should be noted that it is not unlimited at full speed.
Consumer Cellular said:
Consumer Cellular Plans
On unlimited data plans, access to high speed data may be reduced after 35GB of use, and you may experience slower speeds for the remainder of your billing cycle.
 


I reread the relevant passage, and it doesn't even use the word "speed." I think you may be misinterpreting his meaning. Certainly, greater bandwidth effectively equals more "speed", no matter the medium involved, as a coworker (at a telephone company) pointed out many years ago: "A truck full of tapes is a lot more bandwidth than a telephone line." In fact, the same thing is even true today, when raw video footage for network TV shows is actually rushed back to the studio on storage drives by airplane, not sent over telecommunications lines, as it's simply far too much data to transmit quickly.
... It's not just shipping the tapes, it's also getting that data on and off them to where you really want it. Given that current telecommunications systems run at 800Gbps, per wavelength, they are considerably faster than any plane full of LTO tapes, which max out at 900MBps, or only 7.2Gbps. That's slower than the 10Gbps which is the current baseline speed for telecommunications (I could argue that it's moved to 100Gbps now).
 


Ric Ford

MacInTouch
... It's not just shipping the tapes, it's also getting that data on and off them to where you really want it. Given that current telecommunications systems run at 800Gbps, per wavelength, they are considerably faster than any plane full of LTO tapes, which max out at 900MBps, or only 7.2Gbps. That's slower than the 10Gbps which is the current baseline speed for telecommunications (I could argue that it's moved to 100Gbps now).
Let me know how soon I can get 800Gbps service to my office and what it'll cost.

My tape example was from decades ago, when the truck full of tapes was an alternative to sending data over copper (or microwave) phone lines and vastly faster for large amounts of data. (Fiber was just getting going then.)

Today, we're talking about 4K+ raw video capture to SSDs in very time-sensitive network TV series production and having to ship those files on "disk" by air to studios for editing/finishing, because even the latest and fastest telecom links are too slow for these demanding workflow requirements.

As we've discussed extensively in another topic, it's virtually impossible to even do musical collaboration in real time over distance with typical telecom links today, let alone real-time 4K+ video collaboration, let alone full professional video production.
 


Let me know how soon I can get 800Gbps service to my office and what it'll cost.

My tape example was from decades ago, when the truck full of tapes was an alternative to sending data over copper (or microwave) phone lines and vastly faster for large amounts of data. (Fiber was just getting going then.)

Today, we're talking about 4K+ raw video capture to SSDs in very time-sensitive network TV series production and having to ship those files on "disk" by air to studios for editing/finishing, because even the latest and fastest telecom links are too slow for these demanding workflow requirements.

As we've discussed extensively in another topic, it's virtually impossible to even do musical collaboration in real time over distance with typical telecom links today, let alone real-time 4K+ video collaboration, let alone full professional video production.
... As you are half a world away from the telco that employs me, I'm afraid I can't offer you a quote for 800Gbps, but if you have a spare $1M or so, I imagine your local provider could have it installed in about 90 days. :)

My point was that the truck full of tapes analogy always neglected the time taken to read and write the data, so the 'bandwidth' of the truck was data volume over time, and the time was not just the drive time, but the read/write time.

If 1Gbps were enough, and you were in the country where I live, we could have that installed in three days for about US$50/month.
 



Amazon disclaimer:
As an Amazon Associate I earn from qualifying purchases.

Latest posts