news A crowdsourced comparison of real-world broadband speeds has appeared to show that the MyBroadband broadband availability site launched by Communications Minister Malcolm Turnbull last week is significantly inaccurate, with speeds being almost universally below the data produced by the site.
Turnbull launched the MyBroadband website last week as the culmination of a significant study conducted by the Department of Communications into the availability and quality of broadband in Australia. The study was an election promise by the Liberal MP contained as part of the Coalition’s broadband policy unveiled in April 2013 ahead of last year’s Federal Election.
According to Turnbull, the report is the first of its kind to be undertaken by an Australian Government, with data drawn from all major Australian telecommunication carriers. It describes the broadband technologies available as well as the speed that can typically be achieved over each available technology platform.
However, an analysis of real-world broadband speeds has already shown it to be significantly inaccurate. Following the launch of the site, blogger Noely Neate invited readers to submit their real-world broadband speeds to a public document online. In a very short time, the blogger had received over 800 submissions, representing Australians from all around the nation.
While the data is still being tabulated, early charts of the data such as the one displayed above (click the image above for a larger version) appear to show significant variance of the estimated and real-world speeds supplied by participants, especially when it comes to ADSL broadband. Neate herself has the same problem.
“… for me, those figures were not even close, on a good day we get 6Mbps, if it is raining we get NOTHING,” wrote Neate in a blog post on the issue. “Going to [the] Mobile Broadband area it says we have 3G & 4G Coverage? Umm no 4G here mate. So obviously I asked around, put it out there on Twitter & bloody hell, the avalanche of different figures, some even saying that the site said broadband was not available & they could actually get it LOL!”
Another popular blogger, dubbed ‘Wolfcat’, noted that they had a similar issue with the site, criticising its “overly optimistic” ADSL speed listings. “My area is listed at 12.2Mbps. Congrats to those in my neighbourhood getting those speeds, but sure as hell I am not,” they wrote. “Nor does this speed factor in things such as many peoples ADSL slowing down when it rains for example. I get 4.5Mbps, according to this funny thing called Speedtest.”
At the launch of the site, Turnbull acknowledged that the data would not be 100 percent reliable, as it indicative data based on estimate information supplied by carriers about their networks and not real-world testing data. “You may find that the service in your own address is not consistent with this, but this is the closest you can do without testing every single premise in the country,” the Minister said. “Without obviously perfect information, we are getting a very good handle on when broadband availability is the least.”
The Minister said that the Government planned to integrate real-world speed test functionality into the site, which would allow users to submit their own speeds into the site’s database and increase the accuracy of its data.
opinion/analysis
The interesting thing about this crowdsourced data that has come through regarding the MyBroadband site is not that it shows that the Government’s data is inaccurate. That’s to be expected; as Turnbull himself pointed out at the site’s launch press conference, the site uses estimates and not real-world measurements to give Australians a picture of broadband availability in their areas. By definition, it cannot yet deliver totally accurate data.
The actually interesting thing is that so far, almost all of the real-world data submitted is in one direction — worse than the estimates. The overwhelming trend seen from the many samples collected by Neate and her ad-hoc team of statisticians on social media is that real-world broadband speeds are significantly slower than telcos are estimating, with only a few outliers being better.
Of course, just as it’s possible to question the accuracy of the data collected by the Government’s MyBroadband site, it’s equally possible to question the data being collected by Neate and co. The blogger’s survey is hardly scientific, being based on voluntary contributions online. Nor do we know how qualified her assistants are to analyse that data. We’re not talking about a definitive study here. This information is very, very far from being verified and should not be treated as conclusive.
However, none of this invalidates the clear overall, high-level trend we’re seeing coming from the sample collected so far. Australia’s telecommunications industry has long been full of rumours about the poor quality of Telstra’s copper network, the degree to which it is affected by something as simple as rain, and the claim that many Australians currently on ADSL broadband aren’t achieving anywhere near the speeds they have been led to expect.
For the first time, we’re seeing this play out in reality here, with community-sourced, real-world data clashing directly with ‘official’ data from telcos and the Government. That clash has already appeared to reveal a significant gulf between the official version of the broadband truth in Australia and the real-world experience. We hope Minister Turnbull acknowledges this reality as the Government’s MyBroadband site starts collecting its own crowdsourced data.
And we wonder what impact these new data sources will have on the debate about the potential speeds achievable in a Fibre to the Node scenario for the Coalition’s Broadband Network project.
Image credit: Paul Davis
Interestingly, the amount his site is wrong by is probably a pretty reasonable guide on how unsuitable the copper is.
Both Optus and the iinet group have logged, and realtime address matched sync speed data. That this was not asked for is ridiculous.
Well it is using “average” speeds for the area, so yeah it is ballpark. That said it estimates my ADSL at 8.7, I sync just over 8 so it’s not too far off.
Mobile however is different, it says I have 4G availability, which I don’t, I don’t think it’s available on any carrier in my postcode.
And 3G, well take my neighbour over the road who works from home as an example, there are no DSL ports available in the exchange so has to go with wireless, Telstra has no 3G coverage within the house, and Optus only has coverage if you put the receiver in a specific corner and height (I’ve never tried Telstra in my house by I have a similar experience trying to use my Optus hotspot), so really you might as well write off 3G as being viable as well.
Actually its the median, which really doesn’t make much sense.
I agree, using the median doesn’t exactly tell us much – for an extreme example, imagine if results were polarised (extremely high and extremely low, not much middle-ground) then the median would be whichever of those extremes was more numerous by 1. e.g. 24, 24, 24, 2, 2 (median = 24); 24, 24, 2, 2, 2 (median = 2). What good is it telling us what the middle connection’s speed in our area is? I’d be more interested in the upper and lower values.
Using the mean would make more sense, or maybe mode range where results are closely grouped, e.g. whichever of these ranges is most common for connections in your area (Mbps): 0.1-2.4, 2.5-4.8, 4.9-7.2, 7.3-9.6, 9.7-12.0, and so on.
Any way you go though, it doesn’t matter whether they call it an estimate of the median, mean or mode for your area, or an estimate of your premises’ speed, the estimate they’re currently using is obviously hopelessly wrong.
Now keep that chart and we’ll do it all over again in Winter and compare :)
A cynic might say Turnbull is deliberately trying to misrepresent the true situation. Whether misrepresenting over the real situation is in his interests or not would make an interesting debate in itself.
The ADSL quality maps are my favourite part of that website.
https://www.mybroadband.communications.gov.au/upload/documents/ADSLQuality_Sydney.pdf
https://www.mybroadband.communications.gov.au/upload/documents/ADSLQuality_Brisbane.pdf
Why not use this instead
http://www.tpg.com.au/maps/
More accurate ?
At least is does not show my dsl 50% faster than what it really is.
HFC. That’s funny. Pay double the price. hahaha F…
Well umm this was surprising :)
We only did the survey as Twitter was sort of overloading when comparing what My Broadband said as opposed to the likes of ozspeedtest and even taking into account variables like equipment, ISP etc there still should have been a few that were over the My Broadband ‘median’? We never expected that many respondents, it was really just an ‘interest’ thing.
However even with Paul’s cool graphics with the exported data, the comments were really interesting. A lot of people who are not techy (as you can tell from the language) went out of their way to relate their issues, experience & in some cases actually learn how to even find out their speed, so it started a really good conversation in regard to ‘average people’ being left out of the loop & not being heard in this whole NBN conversation.
We hope to have a submission to the Senate Select Committee on the National Broadband Network along the theme of the ‘comments’ (and rain not being included in MyBroadband methodology) along the lines of “It’s not that families don’t care, it is that they don’t understand & no-one is asking them” finished tonight, we will of course put that up on the blog as well, including a download of the exported data :)
So I missed it first time around.
I don’t suppose you are thinking of opening up the survey again?
At the moment, it is very unlikely that mum will put the survey up again. We are still in the process of collecting and collating the data for senate submission.
Interesting idea for the future though, so who knows :)
“You may find that the service in your own address is not consistent with this, but this is the closest you can do without testing every single premise in the country”
How hard is it, once you’ve entered your address, to have a “Test My Speed” button on the MyBroadband website? That way, over time, they could correct the initial estimates and people could see real world speeds by location. Or is that too much of an evidence based approach?
The problem with “test my speed” options are that they are — in the case of higher speed broadband connections — only testing the speed of your WiFi, not the actual broadband speed. Most people use WiFi these days, ergo, results will be skewed to the performance of WiFi.
There are a lot of things the myBroadband site doesn’t take into account, rain being just one of them, bridge taps are another, pair gain is another, etc…
The other thing i found surprising when looking at the whirlpool thread was how the myBroadband site pretty much sticks to its guns with the A, B, C, D, etc… for the services it is showing you should be able to get.
My connection suffers from all kinds of issues, rain being a very good one, rain knocked me out for an entire weekend.
The bridgetap on my connection (which my ISP can’t tell me where abouts it is located because Telstra won’t reveal that information).
In the last 6 months, i’ve had my ADSL port rebooted a total of 4 times because each time it has locked up and the most my ISP can tell me is that because I live rural wa (south-west Bunbury region), Telstra don’t put a high priority on fixing up services and most likely my DSL port has old buggy firmware on it.
Yet somehow, during the times when my router isn’t losing connection (at least 3 to 4 times a day) i manage to get higher than the speed that myBroadband shows (maybe i’m just lucky, i dunno).
What i wonder, and someone on the whirlpool thread mentioned it, was, if they got this information from Telco carriers (most likely only Telstra), why can’t they go to the ISPs themselves and get the real-world speeds or have Telstra do an SNMP type thing on the ports to give the accurate sync rates.
Why don’t the speedtest sites publish their existing datasets?
They don’t have accurate locations of where those tests are being done from, nor under what conditions.
Google/YouTube is working on this: they’re watching how quickly various clients are able to download video streams, and they know exactly how fast they can shove that data down your connection. “No data available in your area” for me though.
The data is accurate to the ISP & IP address range, which is good enough for Google to send a letter to the ISP and say, “Hey, what’s up?” But this doesn’t give geographic correlation.
Google’s project is over here: http://www.google.com/get/videoqualityreport/
An average, allegedly, in my area of 6.87Mbps. From talking to my tech-savvy neighbors, of which there are several, we’re all lucky to get 3.5Mbps.
Doesn’t this show exactly the value of crowdsourced data? Telecoms and govt can only estimate based on what they know, end users can only give data as to what they see in terms of sync/performance.
Getting a single entity to do an effective map of the whole network in Australia isn’t going to work for a whole bunch of reasons.
Why doesn’t the Govt get together with Ookla/speedtest/whatever to determine a easy-ish way to do a acceptable speedtest, and collaborate with the ISP’s, then literally have everyone run the test at some point (/maybe multiple points to prove how leaky the Telstra network is during rain)? ie broadband census. Surely that would be as good as the existing mybroadband data set?
Some thought would need to go into what the obvious points of misinformation would be, but think of how much quicker/efficient a NBN fttp rollout (yes I’m still hoping), could be with actual information?
“Why doesn’t the Govt get together with Ookla/speedtest/whatever to determine a easy-ish way to do a acceptable speedtest, and collaborate with the ISP’s, then literally have everyone run the test at some point … ?”
Because that would be completely against this Govt’s interests…
Happy to be proven wrong though.
You’d be better off pulling the sync speed databases from ISPs, since the ISP tracks both sync speeds and subscriber location. I imagine this is commercially sensitive information though (even an aggregate), so I doubt it’ll ever get released.
Problem with that is that unless you know what you are doing, download based tests like Speedtest don’t give a true representation of line/sync speed. Plenty of things can throw it off (e.g: dodgy wifi link, other users downloading on other devices, programs/windows update downloading in the background etc).
You *could* tell people to plug in an ethernet cable, turn off other devices and make sure programs aren’t running in the background – but you can be sure that most won’t. And all that leaves you with is another set of data which may or may not reflect reality.
That graphic is a lovely illustration of the phrase “Speeds of up to X”.
Thanks Malcolm.
Overestimates is an understatement…
Gee, It is only 99.9 % wrong across the board.
“We think people will only ever need 12 Mbps”
“We think our fully costed CBN will cost $29 billion”
“NBN in Tasmania will cost $80 billion and take 20 years”
They can’t even get their facts straight even when they try. But it’s not lying apparently.
Yes Andrew, Such blatant LNP propaganda – reminded me of Telstras NWAT (now we are talking) site Thodey took down
For the connection off my TW exchange, I supposedly get 8Mbps, according to My Broadband. Reality is 1/4 of that average 2Mbps
I have tried writing a targetted email to my local Federal LNP MP about this issue, I have had NO response to date
Just an update, I looked up my local MP phone number and called & HE actually answered the phone. I have written into his office on the topic of MyBroadband and the current realities in suburbia.
I spoke for about 30 minutes, to his credit he noted lots of issues with telecommunications in our region and he has arranged for Paul Fletcher to come visit in person.
I hope this makes a difference, but I am not hopeful of any quick remedy
My guess is the my broadband speeds are mostly based on an analysis of line length and what you should be able to get, which pretty much ignores the line quality issues and congestion issues. There is also mention of a Telstra sync speed survey from 2011, They also use 1000base for speeds measurements. like the other reports we are getting they lack details for analytic rigor and are mostly a load of marketing(pretty graphs, that make you look good).
Turnbull’s fantasy:
13mbit
Real world:
7mbit (without rain) and 3 – 4 drop outs a day
4mbit (with rain) and 15 – 20 drop outs a day
My exchange is strange, it has 0 ADSL2 ports, not ports available, 0 Ports; but its listed as ADSL2 enabled.
GREAT SITE!
At least it found your house :P
I actually found the information to be dead accurate for my area. Its says 10.22Mpbs and i get about 10.5Mbps (ADSL2). My estate is a newer area (~15 years old) and has had really good quality net (for its distance from the exchange). I think i’m really one of the lucky ones here. The only problem i have every had is exchange congestion.
Just goes to show that the data is purely based on the speeds expected to be achieved under near perfect conditions. Surely they would be able to produce a model based on the age of the copper in that area and adjust expected speeds based on both the distance from the exchange and the age of the copper.
THis is typical Turnbull ,He is a con man ,not to be trusted ! like Abbott say & do whatever it takes to hoodwink the vunerable!.Lying is promoted ,Don,t let the truth get in the way !!
The worst thing I find with the site is not the inaccurate speeds (which are 4mbps higher than what I get) but rather the fact it states many people can receive HFC in their area when that is a complete and utter lie. How they screwed that up when it isn’t available in my whole suburb from any provider, is beyond me. No idea where they are getting their data from.
Good work Renai.
Keep it up.
Appreciated.
*cough*
w/o resorting to expletives…
There is a difference between “inaccurate” and “kool-aid” in a supposedly documented website/report by the government. Even if they are just “estimates”
One has an implied reasonable margin of error and the other is just a plain old fairy tale.
I live in a small rural town in the middle of Tassie.
Funnily enough mybroadband under estimates my speeds. I am 50mts from the exchange and can generally pull about 20mbps. myBroadband says 18.39mbps. So close…ish.
However, and this is the interesting bit. I have checked about 20 other addresses in town from close in, to the far extremities of 2-3km from the exchange. Everyone of those tests return exactly the same result as mine 18.39mbps.
We must be a very lucky little town.
It’s an estimate of the median (how one estimates a median I’ve no idea).
Naturally, there will be (in your area), those above the median (like yourself) and those below the median. By the definition of a median, it should be exactly half each above and below.
What this crowd sourced data set reveals is that the estimate for the median is a load of bullshit, because the number of those below the median looks to be at least 90%. Somehow very few people above the median have posted their results, or more likely the estimate is just plain wrong. The median estimate should be much, much lower.
MyBroadband tracker would be much more accurate if,
instead of saying “The estimated median ADSL speed for your area is (Mbps):”
it said, “The estimated ADSL speed in the top ten percentile for your area is (Mbps):”
What this crowd sourced data set also shows is that there is just simply no way the Coalition government can possibly justify using these estimates of theirs to assist with planning network rollout priority and design.
My connection is on a Telstra Top Hat RIM. MyBroadband tracker says my area’s median connection achieves 22.97Mbps and the quality is A. My connection sync’s at 18.4Mbps and I get peak speeds of 1.5MBps (12Mbps?), and dropouts and slowdowns are an everyday occurrence. How unlucky must I be to be so far below even the guy in the middle!
(Really, though, I’m luckier than most. My point is that I wouldn’t expect many people, even in my area, to have a better connection than I do, so if anything, my connection should be above the median.)
Of course our “Truthful & Transparent Government” could have easily implemented this exact same crowd sourced data, probably even more quickly & probably with a larger data base, perhaps in even less time. This 800,000 premise sample was achieved in how long ? Less than a week ?
But the better option for a Government who only seeks to deceive for their own preconceived goals would be to, as they have, present the data as they want it represented, make promises of an impending crowd sourced data entry which of course will never happen, because by then their contorted results will have already been used to justify their flawed network to nowhere.
Congratulations Turnbull, at least we`re finally seeing what devious practices you`ve been up to for the last six months, how is you 100/40 HFC home internet connection going by the way ??
I don’t think it was 800 thousand.
Fat fingers there 800,000 was a little off the mark ; ) but you get my meaning.
Problem is that one would expect people who don’t get as good as the site says would complain, that is human nature. Those who get better no doubt in the main are not going out of their way to say anything.. Thats the problem with voluntary surveys
Agreed, but it would also show when they compare results to their own graph`s for a given location how inaccurate their database is, don`t forget they`re using only 20,000 homes as a representation for the whole of Australia.
Based on that, even a primary school student could point out that any build data that they attempt to glean from their current data will be flawed, but of course you would use flawed data for a flawed network.
It`s clear that this website is being used purely as a ruse, to make the common not so savvy consumer believe that the decisions they`re making are justified.
If they were truly trying to make factually correct decisions, based on factually correct data, little of what they`ve produced so far would have progressed past file 13.
It’s still a whole lot better than nothing and that is all we got from the Mybroadband site when it could have been so much better. And embarrassing.
The site says my median speed is 14 Mbps, and I get 16 usually, so no complaints here…
I suspect what’s happened here is Abbot has been all too happy to take the data from the telcos at face value, when they have a clear interest in rather optimistic portrayals of their network speeds and copper quality.
I’d also second the idea of a broadband census – not only would this provide a realistic depiction of network speeds throughout the country, but if optional it would identify the areas where it was in greatest demand. (It could also include questions about deterioration in weather and political position on the NBN.)
That said, I don’t believe for a moment that the Liberals would be interested in this unless they were dragged kicking and streaming into it. Politics is about the manipulation of perception, and objective truths are rather hard to spin.
And not to mention two other variables that would not have been included in the MyBroadband analysis. One is the contention ratios to household services applied by ISPs. 300:1 is not uncommon. So time of day and peak periods are not addressed. The other is the quality of the in-home wiring. How many twisted together and poorly terminated wiring points, and so reflection points, might there be between the Telstra premises boundary and the ADSL modem/router?
The only thing that would affect me is if all you people jumped on my beautiful HFC network.
I’m still sad for the rest of Australia and what could’ve been a booming IT sector though..
http://www.speedtest.net/my-result/3332776748
it estimates my speed just below what I get, within 1M. id say that’s pretty accurate
Give the man (or woman) a prize!
Guest, that’s absolutely silly. Please read what it says. It doesn’t estimate your speed. It estimates the median speed in your area. In that regard it is a very poor estimate.
Of course there are going to be people above and others below the median – that’s in the definition of the median. But something like 90% of real world data points are below the estimated median, meaning that the estimate for the median is just plain wrong. It’s nowhere near accurate an estimate of the median. An accurate estimate of the median would have resulted in roughly 50% above and below the estimate.
Why is this significant? First of all it’s misrepresentative of the true situation across the board. Second of all they’re using the same erroneous data in order to prioritise and plan network rollout and design.
Ran out of edit time.
Why is this significant? First of all it’s misrepresentative of the true situation across the board. Second of all they’re using the same erroneous data in order to prioritise and plan network rollout and design. Finally, their assumption appears to be that the copper is fit for purpose in their estimates of the median speed for an area, and that assumption is also the foundation of their FTTN plan, but as their assumptions have been proven hopelessly wrong by real-world data (due to the state of the copper), then the foundation is gone and so the viability of FTTN is put into question.
It is quite possible that the median results listed are correct, median and average are no where near the same thing, and given that line speed appears to drop off exponentially as you move away from the exchange, you would want to see the average line speed for an area listed along with the median to get an idea of the skew of the distribution
Well least it’s partially true, everything came up as “not available” apart for ADSL and of course 4G coverage (actually very limited)
ADSL I get an A grade, however you’re unable to obtain a phone line in my area (5km from the Perth CBD) which makes it a tad hard to get ADSL, after waiting a year I was able to obtain one.
Suggested speed was 5.41, on a good day I sit around 4.2 otherwise it’s less, have 3x properties within streets of where I live and generally the speeds get worse, they are actually closer to the exchange as well.
I don’t see how even the average is right.
On the web site, my address is one of the closest in the green boundary to the exchange. Yet my actual speed, (measured by looking at my modems sync speed) is WELL below the average quoted.
Even if say my cable ran away then back towards me again, 90% of the premises must be further due to the shape of the shaded area.
Not average, median. Probably why it’s not right.
Note that the data set is self-selecting: it will mostly be people who are upset about their ADSL performance who are responding, and thus anyone who gets “over the line” is probably not going to bother complaining because they have nothing to complain about.
I get about a quarter of the speed that is average my area. If you believe what isaid about average speeds, I know of a bridge in Sydney that I would like to sell to you. It’s no wonder this country is going to end up as just one large quarry.
I’ve found after talking to people and remembering back when I first got ADSL2+, it seems the speed they list may be accurate if your copper is in good condition. My speed listed on site is supposed to be 10.22Mbps, my speed I used to get when I got ADSL2+ over 5 years ago or so was around 9-10Mbps. My speed now is around 6-6.5Mbps normal and 5.5-6Mbps in rain.
As people are saying, it seems to completely ignore the state of the copper. If we can provide feedback that is accurate, perhaps they may actually begin to see the quality of the copper in different areas.
I didn’t even use the speedtest website, i nominated my actual line sync speed, and it was still out: 4.9Mbps line sync speed, vs 8.13Mbps on the mybroadband website.
Seems like they’d be just about on target if they took all of their estimates of the median and halved them:
http://i.imgur.com/TefOfDb.png
It estimates I get 11.18Mbps in my area. It’s a fairly narrow area, with AFAIK only one subexchange at the southern end.
I’m lucky to be within throwing distance, so it massively underestimates the 22Mbps I get.
I’m sure I’m not the only one that is getting (significantly) higher than reported.
In my area Tweed Heads West,
the My Broadband site states
“A” for the first part and
“E” for the second part with ADSL at 3.79Mpbs
but all we can get and that includes every address in the green zone is mobile internet, there’s NO ADSL here.
OK I have ADSL 2+ with Telstra, we can get 12Mbps from the line if we are lucky (say once a month at some obscure time when no one else is in). When watching the TBOX or if it is a hot or rainy day and everyone in the area is inside on their computers / phone / smart TV we get 2Mbps. This is every weekend! Happens to all the Neighbours. I think this is the issue – not what the maximum speed your line can attain at 5:30am in the morning when Telstra decides to test the lines everyone is asleep. Australians are screaming for a faster reliable broadband! Why is it so difficult??
hello!,I love your writing so much! share we communicate extra about your article on AOL? I require a specialist in this space to resolve my problem. May be that’s you! Looking forward to peer you.
Comments are closed.