Friday, October 28, 2016

CalSPEED Testing @ Head of the Charles Regatta


Last weekend, River City Rowing Club teammates and I competed in the Senior Master Four race at the annual Head of the Charles Regatta in Boston. While there, I ran several speed tests at the AirBnB where we stayed -- both on the WiFi provided and on AT&T's LTE network. Here are the results:

Head of the Charles racecourse. Test location shown in red point below.



















River City Rowing Club Sr. Master Four: Michael Paiva, Dan Tharp, Paul Crawford, Whitney Powell, me

Thursday, October 27, 2016

Week 3 Mobile Testing Progress - 75% complete as of Oct. 26



Hello Wireless Loops. Goodbye Fiber? (Part 1)



Wireless Loops As Copper Substitute
There have been reports that California's largest local exchange carrier and Connect America Fund Phase II recipient, AT&T California (maps of California's CAF II areas), plans to use wireless loops to deliver broadband to high cost-eligible households -- in other words, use mobile (i.e. "wireless loops") as a substitute for copper telephone lines to offer speeds of 10 megabits per second down and 1 megabit per second up or faster (10/1). How the FCC measures AT&T's progress in deploying wireless 10/1 could greatly influence the level of investment AT&T needs to make. This post analyzes several scenarios where AT&T uses mobile LTE spectrum to deliver wireless loops. Read Part 2 for corrections and comments to this analysis.
(photo credit: Owen Rochte)

Background
Last year, AT&T, Verizon, Frontier, and Consolidated agreed to receive from the FCC, collectively, $105 million annually for five years to upgrade California homes so they can get at least a 10/1 connection to the internet. The federal program that pays for this is called Connect America Fund Phase II (CAF II), and it is funded by surcharges on our telephone bills.

Fiber Optic Reality Check
As we enter the last quarter of 2016, it's time for a reality check. For those of you living in CAF II high cost areas (primarily rural California) who've been holding out hope for fiber optic broadband access to your home, you may be waiting in vain. That's because CAF II doesn't pay enough for fiber to the home, and carriers are not obligated to build fiber to the home. Furthermore, once the CPUC's California Advanced Services Fund runs out of money, there's arguably no funding mechanism, either federal or state, to bring fiber to rural California homes. 

Can Mobile Deliver?
If engineered as a wireline substitute, mobile can deliver 10 megabits per second downstream and 1 megabit per second upstream (10/1) to the home. The question is what will be used as proof that AT&T's 141,000 CAF II-eligible homes have been upgraded to receive that level of service? We know from mobile field tests that mobile speeds are highly variable from one moment to the next, so in order to deliver a reliable 10/1 service, AT&T may need to augment their network to deliver considerably faster speeds on average. Refer to the CPUC's comments to the FCC on GN Docket No. 15-191 regarding the highly variable nature of mobile broadband and the importance of accounting for reliability when measuring broadband speeds.

The "LTE" Indicator On Your Phone Doesn't Mean Fast Speeds
The FCC has made available to the public some of AT&T's mobile broadband deployment data. Unfortunately, speed data are missing. What remains is a file showing areas with access to LTE (4G "Long Term Evolution") service, as well as other, lower levels of service. We know from experience that mobile coverage shrinks as speeds increase, so it's impossible to tell how large a 10/1 service area extends across California.

Three CAF II Wireless Loop Scenarios 
If AT&T is going to use LTE wireless loops (i.e. home broadband access over the mobile network), here are three initial takes on how many CAF II-eligible households are theoretically covered today using various data sets. Based on these scenarios, AT&T has to make either very little investment to offer 10/1 service to their CAF II high cost households, or they have to make a significant investment. The answer hinges on what validation method the FCC uses to ensure AT&T has met its CAF II obligations to offer 10/1 or faster.
  • Scenario 1: FCC 477 Deployment Data - Using the December 2015 deployment data AT&T filed with the FCC, 85% of AT&T's CAF II high cost households have LTE coverage. Note that "LTE" is not the same as 10/1 or faster. However, if the FCC were to use 477 data as proof of meeting CAF II obligations, AT&T could conceivably claim subsidies for most of the 141,000 high cost households without doing any network upgrades.
  • Scenario 2: Interpolated Average Speed Data - Based on interpolated average speeds from Spring 2016 mobile field testing, over half of AT&T's CAF II high cost households appear to have access to speeds of 10/1 or faster.
  • Scenario 3: Interpolated Mean Minus 2 Standard Deviations - Using this stricter standard, where we reduce the average speed at each of the 1,990 field test locations by two standard deviations and create an interpolated surface using those adjusted speeds, nearly all of the 141,000 households appear to remain CAF II eligible. This suggests that AT&T would need to make a significant investment in their mobile network to deliver a reliable 10/1 or faster service to their high cost households.
Conclusion
Of these three scenarios, the answer to how much additional coverage is necessary is likely to be somewhere between scenarios 2 and 3, because scenario 1 is does not rely on a speed threshold, but rather simply the presence of LTE service. "LTE" offers a wide range of speeds, including zero kilobits per second. For the sake of reliability, the FCC should implement a robust mobile measurement program, such as CalSPEED, to verify AT&T has met its CAF II obligations using wireless loops.

Scenario 1: FCC 477 Deployment Data: 85% LTE Coverage


  
Scenario 2: Interpolated Average Speed Data: 57% High Cost Households with 10/1 or Greater


Scenario 3: Interpolated Mean Minus 2 Standard Deviations: No Households with 10/1
 


























Here is what AT&T's LTE coverage looks like based on Scenario 1 (no speeds indicated):

















Here is what AT&T's LTE coverage looks like based on Scenario 3 (speeds indicated in legend):

Wednesday, October 19, 2016

Roadrunner & Wile E. Coyote

This week, we were treated to an amazing photo by tester Steve Crews of a roadrunner with mountains in the background. The team finished Week 2 having completed 46% of the 1,990 locations (see map image below, completed location shown in red).

Average speeds shown in table below are in kilobits per second (Kbps), so divide by 1,000 to get megabits per second (Mbps).

 
































Friday, October 14, 2016

Two CalSPEED Locations in Destroyed City of Homs, Syria


As I've said in earlier posts, back when we launched CalSPEED in 2013, we didn't restrict distribution to only the United States. We added the restriction because we didn't want people running tests all over the world. CalSPEED is designed to be used by California residents so they can understand how their mobile provider is performing. Nevertheless, we occasionally still see tests appearing in unexpected parts of the world.

At the end of 2014, there were three tests performed at geographic coordinates that appear to be in the City of Homs, Syria. A quick look at those locations in Google Earth reveals the destruction of that city from years of war. See before-and-after photos in The Guardian here.

A New York Times' article on the destruction of another Syrian city, Aleppo, estimates the number of displaced people around the world is equivalent in number to the entire population of the United Kingdom or France. It would seem that number includes all displaced people, not just Syrians, but these images of Syria, even from Google Earth, are devastating.

Here's what part of Homs looks like from Google Earth:


Thursday, October 13, 2016

Believe Me, It's Radar Enforced!

My favorite is the "Radar Enforced" road sign collection. Product of a frustrated highway patrol officer? Below the nine photo collage is a cryptic fortune cookie message.


Week 1 Progress Map

The red dots indicate which locations our testers have completed as of the end of last week. The blue dots are locations that will be tested this week and in coming weeks.


Monday, October 10, 2016

Bravo! Photos from Week 1

I am constantly amazed at the geographic diversity of California, and the photos coming in from our field testers are proof. Their photos are wonderful. I'm adding a few of them from Week 1, but feel free to follow their progress on Instagram at cpuc_broadband_testing.












Mobile Internet Getting Faster, But Less Reliable?

Hey Everyone, Good News! Verizon, AT&T, T-Mobile, and Sprint are able to deliver increasingly faster downstream speeds to your smartphone. The question is, how reliable are those speeds? Can you bank on getting 50 megabits per second every time you try to download a large photo to your smartphone?

The graph below shows the maximum downstream speeds measured for each of the four providers over the life of our mobile field testing program. During this multi-year period, we've seen the introduction of LTE networks by all four providers, and just recently Verizon announced the introduction of LTE Advanced, which promises a 50% increase in speeds.
 




 Newer technology = faster speeds.

Those of you familiar with this blog know that faster speeds sure are great, but more important is the reliability of those faster speeds. How likely will I be able to experience the faster speeds consistently?

One metric we collect is the standard deviation. A low standard deviation suggests that the speed measurements tend to be near the mean (average). Conversely, a high standard deviation suggests that the speed measurements are spread out over a wider range of values from the mean. Higher standard deviation suggests wider fluctuations in speeds.

The data we've collected suggest that as download speeds get faster, the standard deviation increases. For one session, you might get 2 megabits per second, and another session, 24.

Here is a graph comparing the mean downstream speeds between Fall 2015 and Spring 2016. AT&T, T-Mobile, and Verizon seem to be delivering faster average downstream speeds compared to six months before.




















Now, let's look at the standard deviation as a percentage of the downstream speed. Standard deviations have increased as a percentage of the average downstream speeds.

What does this mean? It's hard to say, but it appears the likelihood of getting the average speed at a particular location is lower than before. Here is what the average downstream speeds look like after lowering the mean downstream speeds by two standard deviations. Using this approach, we see a drop rather than an increase compared to the previous round of testing.