Testing Your Home Internet Connection

Ever wondered how fast your internet speed is at home? Sure, you can use speed test web sites, but do you know how much your speed varies depending on what sites you visit?

For instance, did you know that going to a site on the east coast from your home on the west coast can slow down your speed by more than 50%?


I have Comcast for my home service, and my plan offers up to 75 megabits per second, which is over 18 times faster than what I get at work (more on that later). The tests that ran over last night revealed that when my connection went to a west coast server, I got as high as 92 megabits per second, but when it went to an east coast server, my speed dropped to around 39 megabits per second. Moreover, the difference in round trip time (latency), is huge. West coast latency averaged 17 milliseconds, while east coast latency was in the mid-80s. Generally speaking, if your latency is below 100 milliseconds, your internet service can support real-time streaming services (assuming other factor such as packet loss and jitter are within range).

As you can tell from the DIY-looking box in the image above, we're still alpha testing the home measurement device, and we hope to have production versions available for volunteers by 2Q 2018.

Here's a summary of my results:

Spring Field Testing Week 4

We're entering the last week of field testing with only the northern part of Central Valley and Orange County. Average speeds appear to be very similar to those from last fall.

Compare the above speeds to those from last fall's:

 

The progress map below shows completed locations in red and upcoming in blue.

Spring 2017 Testing - Week 3

Week 3 Status: Red = completed test locations, Blue = soon-to-be-tested

Coming into Week 4 of Spring 2017 mobile field testing, our drivers have compiled a large collection of memorable photos. Follow them on Instagram at: cpuc_broadband_testing

Here is a sample of some of their work.

CalSPEED Testing in France

I traveled to Montpellier France three weeks ago and bought a prepaid SIM from SFR (France's #2 mobile carrier) for my unlocked iPhone. Montpellier is France's fastest growing city and its 8th largest. Here are the speed test results from CalSPEED from a test I ran near the Place de la Comedie, pictured below. 

As we saw with 3G/HSDPA technology in California, ping times were long, between 342 and 347 milliseconds, and hampered real-time streaming capabilities. Granted, my phone was pinging servers on the east and west coasts of the United States, but if I were using LTE instead of 3G, I would expect the ping times to be in the range of 150-200 milliseconds for CalSPEED.

This was my first time using a prepaid SIM from a local carrier, and, not knowing how much data to buy in advance, I bought 20 gigabytes. I thought that would be fine for two and a half weeks of exploring the Languedoc area. To my surprise, by the end of my stay I had only used just over 1 gigabyte, and that was with heavy daily usage. Rather than data, it was battery life that ended up being the limiting factor.

Montpellier has a large student population and there are many new cafes catering to them. Pictured below is a sandwich board written in English, tongue-in-cheek.

The writer Rabelais was an ordained monk and practicing physician, and he taught for a few years at the Montpellier medical college, one of France's oldest.

Mobile Field Test Data Gets A Face Lift

We're beta testing a new visualization tool for the California Interactive Broadband Map that displays Rounds 6-10 of our mobile field test data. Below are images from the beta site. The graphics are interactive, so you can select different parts of the charts and drill down to more specifics, such as 1XRTT only (shown below).

 

T-Mobile Latest: A “Fire Chicken?”

With the winter holiday season behind us, it’s time to move that unsold inventory, time to repackage those frozen turkeys as “fire chickens” and call them “fresh."

Fierce Wireless reported in August last year that T-Mobile was replacing its Binge-On sponsored data service with something called “T-Mobile One.” The new service includes unlimited data. However, as with any unlimited plan, the big question is, “Unlimited data at what speed and quality?” Unlimited data at 2.5G or 3G speeds is akin to washing your automobile with a teacup.

As I blogged last March, T-Mobile’s Binge-On sponsored data plan let you stream music and video from select content providers without incurring extra data charges, but the only way to get HD-quality streaming video was by disabling Binge-On, because T-Mobile throttled video streaming to 480 pixels (also known as “SD” for “standard definition”).

It’s no surprise that the Binge-On successor, T-Mobile One, by default only delivers SD quality video. If you want HD quality (720-1080 pixels), you have to pay $25 extra. The only difference between Binge-On and T-Mobile One seems to be the latter lets you stream any content (not just T-Mobile’s preferred providers), and it costs more. Also, just to make sure “unlimited data” doesn’t give people crazy ideas, T-Mobile limits tethering to 3G speeds and also says they’ll throttle anyone who streams more than 28 gigabytes per month (that’s a lot of data, by the way).

Image: Read the fine print!

Read the fine print: “On all T-Mobile plans, during congestion the top 3% of data users (>28GB/mo.) may notice reduced speeds until next bill cycle. Video typically streams on smartphone/tablet at DVD quality (480p). Tethering at Max 3G speeds. Sales tax and regulatory fees included in monthly service price.”

Here is an updated image of estimated streaming video quality based on the interpolated spring 2016 field test results. The January 2016 blog post showing estimated streaming video quality using 2015 field test data is here.

Connect America Fund Layer Now Available on CA Interactive BB Map

As a follow up to my November 1 post on AT&T's plans to deploy wireless local loops as part of their obligations under Connect America Fund Phase II (see "Hello Wireless Loops. Goodbye Fiber? Part 2"), the Connect America Fund Phase II eligible areas are now loaded onto the California Interactive Broadband Map. As shown in the legend to the left, four carriers' territories are shown in light blue (AT&T), dark blue (Consolidated), gray (Frontier), and red (Verizon, now also Frontier). To view the areas, go to the map's menu on the right and expand the FCC Data menu. Check the box next to "Connect America Fund Phase II Locations."

Connect America Fund Phase II Eligible Census Blocks by Carrier

CalSPEED iPhone Ver 1.1.1 Now Available on iTunes

 We've made some improvements to CalSPEED with the iPhone version 1.1.1. Changes include:

• Clearer indoor/outdoor slider
• WiFi reminder screen if device is connected to WiFi
• California and Virginia server testing segments indicated at bottom 
• Streaming video quality indicator added to results summary
• Results history shows video streaming and Mean Opinion Score (MOS) estimate for over the top service

New to CalSPEED? Frequently Answered Questions

     Those of you new to CalSPEED often ask how it compares to other speed testing apps. Here are answers to some of the more common questions.     

      

      How Did CalSPEED Begin? CalSPEED was originally funded by a State Broadband Initiative grant from the National Telecommunications and Information Administration. The testing program began in spring 2012 and has now completed 10 rounds. The test program collects not only speed data, but also speed variation, latency, jitter, and packet loss. With these data, we are able to estimate performance for “over-the-top” streaming voice and video service. With ten rounds of semi-annual mobile testing completed, the CPUC has one of the largest public data sets of mobile broadband performance.

      Where Are Tests Performed? The CPUC tests the same 1,990 locations twice a year. The breakdown is 37% urban locations, 56% rural locations, and 7% tribal locations, which were randomly generated. The field test relies on two devices from four major providers (AT&T, Sprint, T-Mobile, and Verizon). 80 TCP tests are performed for each provider, on each device, at each of the 1,990 locations.

      Is 1,990 Locations Enough? Using advanced geo-statistical methods, we are able to  interpolate service characteristics likely to be experienced by a user located anywhere in the state. The CPUC designed the mix of test locations to cover not only urban places where people live and work, but also rural locations where people may be passing through, such as rural highways and state and national parks. All tests are performed along roads navigable by automobile.

      Why Not Test In Every Census Block? Performing field tests in all of California’s 710,145 census blocks would be prohibitively expensive, impractical and unnecessary. For this reason, neither the CPUC nor mobile providers like Verizon, AT&T, T-Mobile, and Sprint perform tests in every census block, but instead use statistical techniques to approximate service characteristics in between tested locations.

     

 Why Two Servers? Most testing applications use only one, generally nearby, server.  This method understates latency and overstates throughput as compared to using multiple, geographically-diverse servers.Testing to a nearby server results in speeds likely to be experienced for applications such as streaming movies, where content is often cached locally due to its popularity.  However, much of the content broadband users access is not cached locally, so CalSPEED tests two two servers -- one in Arlington, Virginia, the other in San Jose, California -- to understand the role of back haul networks in each provider’s delivery of mobile broadband. While using more than two test servers in disparate locations across the globe would be desirable, using both east coast and west coast servers yields more representative results that testing to only one server.

Is This Better Than SpeedTest.net or the FCC's Speed Test? As shown in a study published by Novarum in 2014 comparing Ookla, FCC, and CalSPEED testing applications, results for Ookla and FCC tests tend to be higher because both intentionally select test servers for lowest latency, which tend to be geographically closer. Moreover, Ookla’s test further biases results by discarding the bottom half of upstream results and bottom third of downstream results. By consistently testing to the same two servers, one on each coast of the continent, CalSPEED provides a reliable backhaul performance metric for each of the four mobile providers. Since we began testing in 2012, we have seen the performance (latency) difference between east and west servers decrease.
 
How Else Does CalSPEED Differ From Other Speed Tests? Most speed test applications rely on crowd sourcing. Crowd sourcing has an inherent selection bias of only collecting data from where it is chosen to be used. Where data is collected, it is biased towards who collected it, why, when and where. In contrast, the CalSPEED methodology has testers return to the same location every time, and the geographic distribution of test locations provides a more complete picture of mobile broadband across the state.

How Many TCP Threads Does CalSPEED Use?  Multi-threading means opening more than one connection to the host and combining them in order to boost overall throughput and is used by many speed test applications. When the CPUC designed CalSPEED, we examined the effect of using multiple threads (“flows”) and concluded there was no material difference in mobile throughput between four threads versus eight threads or sixteen threads. The current test design has 4 threads, each divided into ten 1-second tests for upstream to the west server, then again to the east server. The same is true for downstream. This is then repeated a second time, totaling eighty 1-second tests. Most applications only use one thread. 

Why Do Speed Testing? Carriers Already Have Coverage Maps. Most provider maps show a single coverage color and say things like "4G/LTE Coverage." Through CalSPEED, the CPUC has been able to discern more subtle speed and coverage differences by region. Some providers advertise speeds, but we have observed that those speeds are not ubiquitous, that is, they are not available everywhere providers claim to offer service. Speeds vary widely depending on if you are in an urban, rural, or tribal location. For this reason, we create a heat map of speeds based on actual field test data, and the heat map shows how speeds vary across the state. 

Why Not Use Average Speed, Like Mean or Median? The CPUC has demonstrated[1] through years of methodical field testing that mean and median speeds, by themselves, are unreliable indicators of what consumers can expect to experience reliably at a location. CalSPEED takes observed variability into account to determine speeds that consumers can consistently expect to receive.  As mean throughput increases, so does the amount of variability around the mean.

---

[1] See Section 2.4 Intra-Session Variation in “CalSPEED: California Mobile Broadband - An Assessment - Fall 2014,” by Novarum.

Complete! Fall 2016 Mobile Field Testing

Thank you to the field test drivers from Cal State University, Chico and Cal State University Northridge for finishing this 10th round of mobile field testing. Thanks also to CSU Chico's Geographical Information Center and CSU Monterey Bay's School of Computing and Design for their continued support for this innovative project.

As always, we will be posting summary data on the CPUC's website once it becomes available.

Hello Wireless Loops. Goodbye Fiber? (Part 2)

Last week, I posted three scenarios examining AT&T's mobile LTE coverage of high cost-eligible homes under the FCC's Connect America Fund Phase II program. That analysis assumed AT&T would be using their existing mobile network and licensed LTE spectrum...

Turns out, that was a rosy assumption.

Since last Friday, I received more information from Steve Blum about AT&T's plans to deploy wireless local loops (thanks, Steve!). Rather than revisit the issues, I want to make some corrections and comments to my previous post.

1. AT&T plans to deploy a new technology for wireless local loops whose "success in the marketplace is thus unproven." (paragraph 51 of their SEC 425 filing). For more, see #1 below.
2. Rather than use licensed LTE spectrum, which includes the incredibly valuable 700 MHz band, it appears AT&T will be using a higher frequency (2,300 MHz, or 2.3 GHz), which requires line of sight between cell tower and customer location. For more, see #2 below.
3. Other risks include spectrum constraints on the number of subscribers who can use the service simultaneously during peak usage, and this "reduce[s] the prospects for a successful rollout by AT&T of fixed WLL as a standalone product." For more, see #3 below.

1. Did AT&T Say They'll Need More Cell Sites? 

Group President and Chief Strategy Officer John Stankey's testimony, which is part of AT&T's Form 425 filing for the DIRECTV acquisition, states: "A fixed [wireless local loop] service requires substantial upfront investments. AT&T must install additional antennas and other equipment at each cell site in areas it seeks to serve." (paragraph 50). 

There's no mention of needing more cell sites. 

Modifying existing cell sites is one thing. Acquiring, permitting, and building new sites is completely different. As one El Dorado County supervisor says, new sites are very controversial because of their "aesthetic downside." Aside from having to find property and possibly develop it, there's the sticky problem of getting community buy-in for more cell towers. 

If AT&T plans to use a higher frequency like 2,300 MHz for wireless, they will need to add cell sites. That takes time. Finding additional sites, much less building new ones, will have a significant impact on AT&T's ability to deploy wireless local loop services in a timely manner. It also puts more pressure on local governments to expedite site approvals. 

How many new sites are needed? To get an idea, here is AT&T's presentation to the El Dorado County Board of Supervisors. AT&T estimates they'll need to modify 4 cell towers and add 34 new locations -- either collocated on existing buildings or newly constructed cell sites. 38 locations seems insufficient to serve all of El Dorado County’s high cost-eligible CAF II households, so maybe this is the first phase in AT&T’s five-year rollout. 

2. Frequency Matters 

There are solid engineering arguments why 700 MHz is more valuable than 2,300 MHz for delivering wireless broadband to homes. I’ll address only the line of sight argument. The larger-wavelength 700 MHz frequency penetrates buildings and foliage better than the shorter-wavelength 2,300 MHz frequency. For 2,300 MHz service to reliably deliver broadband, there needs to be line of sight, meaning you can physically see, without obstruction, between cell tower antenna and customer antenna. Without line of sight, service will be less reliable than with 700 MHz. Many parts of rural California are mountainous and have trees, which makes line of sight broadband difficult and expensive to deploy.

Home in El Dorado County with Trees Obstructing Line of Sight

  

Diagram showing line of sight wireless broadband. No obstructions allowed.

Diagram showing non-line of sight wireless broadband, e.g. 700 MHz

 

Credit: L-Com Connectivity, www.L-com.com 

3. Is Wireless Local Loop Service Viable Without Video? 

AT&T’s Form 425 testimony mentions the importance to their revenue stream of bundling broadband (through WLL, U-Verse, or their fiber product, GigaPower) with video (through DIRECTV), voice, and even mobile. As John Stankey stated, capacity constraints reduce the prospects of WLL as a viable standalone service. Put another way, WLL might be financially viable only if a subscriber opts to subscribe to video and/or mobile as well.

The prospect of adding new satellite and cable television subscriptions is dubious in today’s marketplace. One study claims that more than 40% of satellite and cable subscribers are planning on cutting back or dropping their pay TV service. Broadband data from the CPUC's 2013 annual report on the Digital Infrastructure and Video Competition Act indicates that California's state video franchise holders now provide more broadband service than video service.

That doesn’t bode well for AT&T’s DIRECTV subscribers, some of whom may be switching to over-the-top services like Netflix or Amazon Prime and require only a reliable internet connection to watch shows. To underscore that point, AT&T's Form 425 filing for the DIRECTV acquisition includes an economic assessment, which states "higher Internet access speeds will erode the traditional advantage that DIRECTV's satellite network has given it" over video on demand providers like Netflix. (page 54)

More on AT&T’s proposed WLL broadband service (from the El Dorado County video):

• Broadband speeds must meet or exceed 10 megabits per second down and 1 up
• Latency should not exceed 100 milliseconds
• Initial minimum usage allowance of 150 gigabytes per month
• Monthly rates must be comparable to fixed wireline urban rates for the same level of service in urban areas or at or below the FCC Annual National Rate Benchmark (approx. $72/month)
• AT&T must meet specified deployment milestone dates by state
• 40% built by year-end 2017
• 60% by year-end 2018
• 80% by year-end 2019
• 100% by year-end 2020-21

Week 4 Testing: 82% Complete

Our testers are nearly finished with the 10th round of statewide field testing with 82% complete as of last Friday. Here's a map of the completed locations in red, and those still needing to be tested in blue:

CalSPEED Testing @ Head of the Charles Regatta

Last weekend, River City Rowing Club teammates and I competed in the Senior Master Four race at the annual Head of the Charles Regatta in Boston. While there, I ran several speed tests at the AirBnB where we stayed -- both on the WiFi provided and on AT&T's LTE network. Here are the results:

Head of the Charles racecourse. Test location shown in red point below.

River City Rowing Club Sr. Master Four: Michael Paiva, Dan Tharp, Paul Crawford, Whitney Powell, me