A New way To Compare ISP
A Speed Test is a test to measure the access performance metrics, such as connection data rate and latency.
By the test it is possible to measures the bandwidth (Downlink/Uplink Speed) and latency assign form an ISP to his customer. It is an E2E test from a source (mobile, laptop…) versus a dedicated and optimized Server. Each test measures data rate for the download direction, i.e. from the server to the user computer, and the upload data rate, i.e. from the user’s computer to the server. The tests are performed within the user’s web browser or within apps.
Any Speed Test software uses a custom protocol for communication between servers and clients.
Bandwidth, Communications Speed or Connection Speed
A data connection, bandwidth, communication speed or connection speed is the total maximum transfer rate of a network connection. Essentially, it is a measurement of how fast data can be sent over a wired or wireless connection, usually measured in bits per second (bps).
Most broadband connections are asynchronous, which means there are different speeds depending on the way data is traveling:
- the download speed is how fast a client can gets files from Internet
- the upload speed is how fast a client can sends files to Internet
Download speeds are nearly always faster than upload speeds with these connections.
SPEED TEST Services over Internet
Over recent years, customers are always more careful about quality of offered services. This generated a deeper attention about fruition of acquired services. Consequently, a new rivalry has been born between companies, about offers to their customers and the way to measure their Customer Experience (CEM – Customer Experience Management). Numerous third-party web pages have been designed to test an Internet connection speed.
As follow there is a short list of some web services used to test the connection speed & parameters:
OOKLA – SPEEDTEST.NET
The most commonly use SPEED TEST has been designed by OOKLA. OOKLA created a web service that provides free analysis of Internet access performance metrics, such as connection data rate and latency.
Ookla solutions have been adopted by nearly every Internet Service Provider in the world, and have been translated into over 30 languages for use by thousands of small businesses, federal and state governments, universities and major organizations such as AT&T, BBC, Cisco, Comcast, FCC, Reuters, Time Warner, Verizon, Vodafone and Vonage. Over three million people a day use Ookla software, making the company the dominant leader in broadband connection testing.
Founded by Internet and telecommunications veterans in 2006, Ookla has offices in Seattle, WA. – Rif. ookla.com/about –
How speedtest.net works?
First of all let’s try to deeply understand how Speedtest basically works. Speedtest.net, by OOKLA, operates mainly over TCP Testing with a HTTP Legacy Fallback Testing for maximum compatibility. Speedtest.net measures ping (latency), download speed and upload speed.
TCP Test Components
Latency/Jitter
- This test is performed by measuring the time it takes for the server to reply to a request from the user’s client. The client sends a message to the server, upon receiving that message, the server sends a reply back. The RTT (Round-Trip-Time) is measured in ms (milliseconds).
- This test is repeated multiple times with the lowest value determining the final result.
Download
- The client establishes multiple connections with the server over port: 8080. The client requests the server to send an initial chunk of data.
- The client calculates the real-time speed of the transfers, then adjusts the chunk size and buffer size based on this calculation to maximize usage of the network connection.
- As the chunks are received by the client, the client will request more chunks throughout the duration of the test.
- During the first half of the test, the client will establish extra connections to the server if it determines additional threads are required to more accurately measure the download speed.
- The test ends once the configured amount of time has been reached.
Upload
- The client establishes multiple connections with the server over the defined port and sends an initial chunk of data.
- The client calculates the real-time speed of the transfers and adjusts the chunk size and buffer size based on it to maximize usage of the network connection, and requests more data.
- As the chunks are received by the server, the client will send more chunks throughout the duration of the test.
- During the first half of the test, the client will establish extra connections to the server if it determines additional threads are required to more accurately measure the upload speed.
- The test ends once the configured amount of time has been reached.
HTTP Legacy Fallback Testing
Latency/Jitter
- This test is performed by measuring the time it takes to GET a response for a HTTP Request sent to the web server
- This test is repeated multiple times with the lowest value determining the final result.
Download
- Small binary files are downloaded from the web server to the client to estimate the connection speed
- Based on this result, one of several file sizes is selected to use for the real download test
- The test is performed with cache prevention via random strings appended to each download
- Throughput samples are received at up to 30 times per second
- These samples are then aggregated into 20 slices (each being 5% of the samples)
- See “How Results Are Calculated” at the bottom for more details
- The remaining slices are averaged together to determine the final result
Upload
- A small amount of random data is generated in the client and sent to the web server to estimate the connection speed.
- Based on this result, an appropriately sized chunk of randomly generated data is selected for upload.
- The upload test is then performed in chunks of uniform size, pushed to the server-side script via POST.
- We’ll use up to four HTTP threads here, as well, to saturate the connection.
- Chunks are sorted by speed, and the fastest half is averaged to eliminate anomalies and determine the result.
Deciding the number of threads
Speedtest.net will use up to four HTTP threads during the download and upload portions of the test. However, it will only use more than two threads if they are needed to accurately measure the speed, so as to minimize the effect of HTTP overhead on lower-speed connections. After the pre-test, if the connection speed is at least 4 megabits per second, then Speedtest.net will use four threads. Otherwise, it will default to two threads.
However, there is a hurdle on older browsers: Internet Explorer 7 (or earlier) and Firefox 2 (or earlier), the browser strictly adheres to the HTTP specification of only two threads per hostname. To scale up to four threads in these older browsers, it is a must to open the third and fourth thread to a secondary URL provided by the host that points to the same server. I this way, OOKLA Speed Test can works around the limitations of those older browsers and still measure higher-speed connections accurately. Most of OOKLA hosts has a working secondary URL, but if the test has performed from an older browser to a host that doesn’t, Speedtest.net will be limited to two threads at maximum.
This is one reason why OOKLA recommends that all visitors use up-to-date browsers
How results are calculated
All samples are sorted by speed. The two fastest results are removed and the bottom 1/4 which is left (which is approximately 22% of the total). Everything else is then averaged.
- Download Throughput Samples
- Download Throughput Samples Sorted
- Average Calculation Schema
Despite the amount of tests daily performed by OOKLA, there is still an open question: “What is the best method to measure an ISP’s performances?”
Does the type of speed test make a difference?
Based on our experience, HTML5 speed tests are more accurate than tests performed with Java and Adobe Flash. Furthermore, in a customer perspection view, multi-thread tests such as those used by Ookla (Speedtest.net and branded by many ISPs1) don’t represent a real-world network traffic as well as single-thread tests.
In fact, with a multithread speed test it is possible to reach the maximum network performance and measure the maximum Downlink/Uplink Throughput but not necessarily this will be the average Throughput felt by a standard customer.
Any speed test alone do not solve the issue!
In order to optimize connection issues, it is necessary to have:
- the ability to act quickly with collected information, that have to be organized in a single analysis interface
- analyze all time-stamped data in order to understand also the right timeslot when the issue happened
- receive alarms and view a behaviour KPI history using your desktop or handheld device
- deploy your team armed with the information they need to pinpoint the network problems
This means that ISPs need a professional tool to pinpoint and troubleshoot the network issues.
We have solutions that can help any ISP. We talk about solutions and not solution, because any problem can have different consequence on offered services.
e.g. If you think about one OTT (Over The Top). Let’s say YouTube; in a specific position, you can have great OOKLA’s measured KPI but the video’s quality could be awful (too many rebuffering, too much buffering time,…). This because, just as example, the packet loss could be too high or the service itself could have some issue (in fact, in order to rate the connection’s reliability, it is a need to perform statistics also about packet loss, and packet retransmission).
It Is Not Enough!
It should be clear that Speedtest.net is basically measuring only the an internet connection’s speed and in a professional view it is not enough to understand the connection quality itself. An ISP could use a low speed as problem identification. As a consequence of this, it will start immediately to monitor the connection using professional tools to collect all details and protocol messages. Therefore, many ISP decided to use a dedicated and proprietary speed check tool. As example, we are successfully proposing new a Speed Test based on dedicated servers and a proprietary logic.
Another example, is the Italian Authority that created their own dedicated project to test any network connection. Collected logs and measured values can be used as evidence in cases where the user wishes to exercise the right of withdrawal, compared to contractual promises of Internet access speed is not maintained by the ISP.
The AGCOM project servers in fact, are positioned to the NAP (Neutral Access Point), the physical interchange points between different network operators. The particular location of these devices ensures that the exchange of data, real object of the measure, only transits through the network operator’s liability and not on other networks. The measure is based on an exchange of packets between a client, it consists of your PC, and a server located in the NAP neighbor. On the basis of this transmission are measured established KPI. Each user will point the measurement server next to its NAP. The location of measurement points based on these criteria, together with a series of guarantees on the conduct of the tests, makes the certified measurement. This measure with Ne.Me.Sys. (Network Measurement System) lasts for a maximum of 3 days and not only 10 secs. This should make us think about differences between a Professional Dedicated Service and a Commercial one.
Bandwidth Monitor Utility
Another great way to test an Internet connection is to install a bandwidth monitor utility. This software will continuously monitor the bandwidth and will report the connection speed average by day, month, or year.
There are different software solutions to help monitor and list the total data transferred over your computer.
Than…
We always propose the best and most complete results of an end to end test, instead of the subjective test of speedtest.net, which only measures the bandwidth (up and down) for the partial connection between devices and unknown server on the internet. Some examples, and more details, of our use cases could be provided under request.
To our opinion regular measurements will be the best way to understand the behavior of any connection. Especially when customer are using mobile broadband connections in different places, a snapshot does not make any sense in qualifying the quality of the services provided by the operator. In case of fixed connections more stability can be assumed. However, the connectivity to a service has also more elements included like:
- end device, like computer and local network (Wifi!!)
- shared connections in last mile (e.g. overbooking in DSL)
- ISP network
- location if the server which provides the server
- utilisation of the server
To take all these elements into account an individual testing might not end an objective qualification of a provided service, which might need continuous testing by an independent party reporting the overall services and network quality of the operators/ISP. This is exactly where we could help as an independent (third) party.
So, in order to measure an ISP network connection, ask to a professional company that could helps you with a professional and dedicated solution based on the special use case!
Ask to Us! We will be more than happy to analyse your problem and propose you the best custom solution.
Notes:
Many ISP offers their own speed tests and many of it provides it by Ookla. Not surprisingly, the highest consistent speeds you can achieve will be always by these ISP own Speed Tests.
In cooperation with Arjen VAN TRIGT from Tulinx.com
