Monday, June 29, 2015

Fi Comes to CalSPEED


We were happy to see the first Fi tests run on CalSPEED last week. Fi is one of  Google's latest projects, and it combines Sprint, T-Mobile, and WiFi networks to provide seamless service. Click here for coverage of the April 2015 announcement by Google.

Here's a summary of last week's tests:


Thursday, June 18, 2015

Backhaul Sleuthing in Rural California


As part of our investigation into why rural mobile performance is so much worse than urban, we had our testers run trace route tests to see where each provider sent traffic on its backhaul network. We just received the data output, and it's over 36 megabytes in comma delimited format. More than 16,000 tests, some with more than 20 "hops." It will take some time to decipher everything and draw conclusions, but one thing seems apparent immediately: AT&T has fewer hops than the other providers to get to the same destination. This may be one factor in AT&T having higher overall VoIP capability than the other providers (fewer hops, lower latency). Here is a table of the average number of hops to the West server for each provider:


Tuesday, June 16, 2015

Alpha testing of CalSPEED for iPhone begins now

We have just begun alpha testing the iPhone version of CalSPEED. The screen shot above, taken from an iPhone, shows that there's still work to do. For example, users will not need to enter a location ID (as our field testers have to do when we do mobile testing). The second image below is a draft version of the user interface that is being developed.





Monday, June 15, 2015

Spring 2015 Field Testing Complete!!


Our intrepid team of mobile field testers from Cal State Chico and Cal State Northridge have finished another round of mobile field testing, covering around 1,990 locations across the state. This is the first time we tested Android tablets in the field, and it significantly reduced testing time not having to swap out data cards on a netbook. Some of that time reduction, though, was taken up by a new traceroute test that the testers ran so we can begin to get a better look at the backhaul networks.

Results are still coming in and will be cleaned, compiled and analyzed over the coming months.

Thank you to everyone who helped make this a success!

- photo courtesy of Steven Crews

Tuesday, June 9, 2015

Broadband Testing At Mojave Wind Farm

This Vine reminds me of how California is leading the country in adopting renewable energy. California's Renewables Portfolio Standard (RPS) began in 2002 and is one of the most ambitious renewable energy standards in the country. The RPS program requires investor-owned utilities (IOUs), electric service providers, and community choice aggregators to increase procurement from eligible renewable energy resources to 33% of total procurement by 2020. Read more about RPS here.

Actual RPS Procurement Percentages in 2013
PG&E- 23.8 %
SCE – 21.6%
SDG&E - 23.6%



Friday, June 5, 2015

What Good Is A Coverage Map?

Have you ever gone to a mobile provider's web site or their store to see their coverage map? You'll notice the maps never show speeds. There's a reason: the maps are generally based on signal levels, not speed. Instead, providers say things like "4G LTE coverage," which means nothing to the average user. If you don't have time to read this all the way through, skip to #3 below.

As part of our state-wide mobile testing, we have started to peel back the layer of coverage and ask, "What does coverage really look like when we use actual speed?"

Here is a quick comparison of a single provider's coverage, 1) as advertised, 2) as measured, in terms of average speed, and 3) as adjusted by subtracting one standard deviation from the average (mean) speed.

1. Advertised Speed.
Here is one provider's maximum advertised downstream speed for California. The bright green indicates a downstream speed range of 10-25 megabits per second. The lighter green is 6-10 megabits per second, and the yellow represents a range from 768 kilobits per second to 1.5 megabits per second.


2. Measured Speed.
Here is the same provider's downstream speed based on test results at 1,990 locations across the state. Results from each location were spatially interpolated using a method called "kriging" to create a continuous surface. In this case, the bright green areas are significantly smaller, and more colors have appeared, representing slower speeds: 1.5-3 megabits per second (orange), and 3-6 megabits per second (brown).


















3. Mean Minus One Standard Deviation.
Here, we adjusted the measured results at each of the 1,990 locations by subtracting the mean value by the standard deviation for the location where the measurement was performed. We do this, because we have observed that the higher the speed, the larger the standard deviation. In some cases, the standard deviation can be as high as 30% of the mean. This adjusted coverage map may likely be a more realistic picture of your mobile provider's coverage.
















4. Higher Speed = Higher Variation
How large is the average standard deviation? Below is a chart showing how large the standard deviation can be when averaged across all measurements for urban, rural, and tribal. Graphs are for both upstream and downstream.

Upstream













Downstream

Friday, May 29, 2015

Reality Check: Sprint #1 Network in Two Years?

Sprint CEO Marcelo Claure predicted Sprint will have the top network in the next 18 to 24 months. Claure said that when he started, Sprint's network was "drop-dead last." We decided to see where Sprint has come from and how far it has to go to beat AT&T and Verizon in network performance. These data compare results from tests performed twice a year for smartphones and netbook data cards. The first three rounds of testing were done for 1,200 locations, the second three rounds expanded to 1,990 locations.

Device: Samsung Galaxy 4 (LTE)

1. Overall TCP Failure Rate

Looking at how often a TCP connection fails (TCP is the measurement used most often when comparing speed), data from tests of all four providers showed a slight increase in TCP failure, with T-Mobile and Sprint on the higher end.
















2. Locations Where TCP Met "Served" Threshold

For purposes of comparison, we looked at the percentage of tested locations where average TCP downstream met or exceeded 6 megabits per second and average upstream met or exceeded 1.5 megabits per second. This threshold is what we use at the CPUC for determining "served" status. Note that these averages are not adjusted using standard deviation. "Average" in this case averages smartphone & netbook as well as west and east coast servers.














3. Over The Top VoIP Readiness

Using a Mean Opinion Score formula we reported on earlier in this blog, we calculated the number of measurements (1 measurement = 1 location/device/provider combination) where the calculated Mean Opinion Score met or exceeded 4.0.