The Measuring Broadband America (MBA) platform collects Internet performance data from many millions of homes simultaneously. There are currently over 500,000 homes safely reporting anonymized data. The test agents, which can be either Whiteboxes or routers, run measurements against popular and important services, including streaming services, emergency services, and e-commerce companies, as well as conventional speed tests and other lower-level performance measurements. The data is then pushed to the secure cloud-based platform and made available in the MBA dashboard for real-time analysis. The test methodology is open, published, and constantly reviewed for improvements. New tests, for example, are frequently added to the program.
The causes of poor performance can appear anywhere on the Internet, and while conventional web-based speed tests can indicate whether your Internet connection is “good” or “bad”, they do not help identify the root cause of issues. With more network segmentation, it is possible to identify the origin (and therefore the cause) of poor performance. The MBA methodology provides for a test agent to be located on the edge of the home network. This can either be a SamKnows-enabled router or dedicated hardware, such as the Whitebox.
The MBA test agents run tests against on-net servers, off-net servers, and cloud-based test servers. Off-net servers are all dedicated to the MBA program and monitored closely. Alongside the data collected from these measurements, a separate and extremely large control panel of SamKnows-enabled routers is maintained. This consists of several hundreds of thousands of homes and is growing fast. This data is used to check the smaller MBA panel to ensure that the results from these homes are not out of the ordinary. This control panel is important to ensure high standards of accuracy.
The MBA test platform supports a large variety of tests, many of which are used in the Measuring Broadband America program. All tests are written in C, a low-level language that enables us to write high-performance measurements with minimal resource usage. More tests are being developed over time so that the MBA can keep up with popular content and measure the most popular services.
The Whitebox can detect 'cross traffic' in the home; this is when other devices inside the home are also using the Internet. Naturally, we do not want to carry out measurements when the user is using their Internet connection because (a) we might disrupt their Internet experience and (b) our measurement results would be impaired.
The threshold for cross traffic is defined as 64kbps downstream and 32kbps upstream. If cross traffic is detected when a scheduled test is due to execute then the test is deferred for 30 seconds and retried. If cross traffic persists for five minutes then the test is aborted.
The speed tests use eight parallel TCP sessions and run for 10 seconds. An additional warm-up period is used to offset the ramp-up period associated with TCP applications. For more detail on how the speed tests work, please see this page.
The latency and packet loss tests run continuously in the background, trickling out a steady stream of approximately 2000 UDP echo packets each hour. This provides a much more detailed and realistic view of latency and packet loss than web-based tests and mobile apps, which just measure latency for a few seconds. Our latency and packet loss test also captures disconnection events - when your broadband connection drops completely. For more detail on how the latency and packet loss tests work, please see this page.
The web browsing test measures how long it takes for a popular web page to load on the user's broadband connection. A selection of nine real websites are used in this metric. Note that this is an example of a test that runs to a real content provider, rather than a synthetic test server. Please see this page for more information on this test.
The test platform also runs a variety of other tests for DNS performance and failure rate, CDN performance, voice-over-IP emulation, and more. Please see this page for a more detailed description for each test that is supported and the metrics that each test captures.
The Whitebox runs tests according to a pre-defined test schedule. This schedule has been crafted to ensure that the FCC can carry out all of the measurements it needs to fulfill the reporting goals, while not using too much of the user's bandwidth.
The test schedule generally runs more measurements during peak hours (7-11pm local time), as this is the primary focus of the Measuring Broadband America report. Some tests will run more or less frequently than others, depending on the reporting goals and the amount of data they consume.
The most recent test schedule is visible in the Measuring Broadband America technical appendix, available here.
Whitebox users are also able to go to SamKnows One to initiate an Instant Test from their Whitebox. These real-time tests allow users to have an instant view of their broadband performance. Please note that the results of instant tests are not used in the published Measuring Broadband America reports; only scheduled tests are used.
The Whitebox is a small Linux-based hardware device that volunteers are asked to install in their home.
Using a hardware test agent provides a number of benefits that help ensure we are gathering the most accurate measurements possible:
All measurements carried out by the Whitebox are 'active' measurements - i.e. it generates its own traffic and measures performance characteristics of that. The alternative is 'passive' measurements, which is when a user's existing Internet traffic is monitored in order to determine performance. The Whitebox does not use passive measurements.
To read more about the Whitebox, please see this page.
The Measuring Broadband America program uses a large number of test servers. These are used as the endpoints for many of the tests (but not all, as some run to real content providers).
Test servers are installed in nine major peering locations around the U.S. These are:
All test servers are hosted 'off net'. This means that they are not installed inside ISP networks, and instead sit outside of the ISP networks at a neutral location. This is done to ensure that we are measuring the complete service that the ISP is providing, up to and including a common handover point to the wider Internet.
The vast majority of test servers have 10Gbps or even 100Gbps. Those locations that are still 1Gbps are being upgraded to 10Gbps over time.