Further evidence Turnbull’s MyBroadband tracker overestimates speeds

38

labor-chart


news The Australian Labor Party has published what it claims is further evidence that the MyBroadband broadband availability site launched by Communications Minister Malcolm Turnbull in February is significantly inaccurate, with average broadband speeds in the Federal electorate of Perth universally below the data produced by the site.

Turnbull launched the MyBroadband website in February as the culmination of a significant study conducted by the Department of Communications into the availability and quality of broadband in Australia. The study was an election promise by the Liberal MP contained as part of the Coalition’s broadband policy unveiled in April 2013 ahead of last year’s Federal Election.

According to Turnbull, the report is the first of its kind to be undertaken by an Australian Government, with data drawn from all major Australian telecommunication carriers. It describes the broadband technologies available as well as the speed that can typically be achieved over each available technology platform.

However, an analysis of real-world broadband speeds quickly showed it to be significantly inaccurate. Following the launch of the site, blogger Noely Neate invited readers to submit their real-world broadband speeds to a public document online. In a very short time, the blogger had received over 800 submissions, representing Australians from all around the nation.

At the launch of the site, Turnbull acknowledged that the data would not be 100 percent reliable, as it indicative data based on estimate information supplied by carriers about their networks and not real-world testing data. “You may find that the service in your own address is not consistent with this, but this is the closest you can do without testing every single premise in the country,” the Minister said. “Without obviously perfect information, we are getting a very good handle on when broadband availability is the least.”

The Minister said that the Government planned to integrate real-world speed test functionality into the site, which would allow users to submit their own speeds into the site’s database and increase the accuracy of its data.

Neate and a group of other volunteers conducted analysis on the data collected and published an extended document submission to the Senate Select Committee on the National Broadband Network. The vast majority of those making submissions (688 out of more then 800) were using ADSL broadband, the most popular form of broadband in Australia. Of those, analysis conducted by the group showed, most participants experienced real-life speeds more than 25 percent slower than the estimates generated by the MyBroadband site.

Last week, in the lead-up to the Western Australian Senate by-election, the Opposition published a second, much smaller study, which appeared to show the same inaccuracy trend in the MyBroadband tracker.

Federal Member for Perth, Alanna MacTiernan, said in a statement that she had surveyed nearly 200 homes in her electorate and found “disturbing” differences between the Internet speed that the MyBroadband tracker claimed was available and the actual speed that locals could get access to, as shown in the chart above.

“Mr Turnbull is deluding himself about the true state of the copper wire,” MacTiernan said. “A new survey of locals around Perth has shown the dilapidated state of the copper network that the NBN is now forced to rely upon.” The Coalition’s Broadband Network policy will involved extensive reuse of Telstra’s copper network as part of the use of the Fibre to the Node rollout style, which is technically inferior when compared to Labor’s Fibre to the Premises model.

“Our survey shows that the actual speeds in all suburbs are far less than Mr Turnbull’s claim in his Broadband Quality and Availability Review,” MacTiernan added. “Residents in many areas across the electorate report that even phone services over the copper wire are unreliable, and what limited internet is available drops out.”

Shadow Communications Minister Jason Clare said the Coalition was creating two classes of people — those that will get “the real NBN” and those that wouldn’t. “Under Labor all homes and businesses in Perth would have received fibre to their premises. Now only some homes and businesses will get this,” he said. “Everyone else will have to rely on the second rate old copper system. The NBN is now effectively dead. Some people will get the real NBN, others will get the second rate old copper system.”

Shadow Assistant Minister for Communications Michelle Rowland added: “Tony Abbott seems to be stuck in the past. He wants to bring back knights and dames, but refuses to invest in things like the NBN which will set WA up for the future.”

opinion/analysis
I think it’s important to acknowledge here that this data supplied by Labor is hardly scientific. We have absolutely no information at this point about how this data was collected or tabulated. And, of course, Labor has an obvious incentive to publish data which pushes its political aims. In this sense, I would regard it as quite suspect.

However, in a general sense, this isn’t what matters. It’s not the specific results in a certain suburb or even electorate which will illustrate the overall usefulness of Turnbull’s MyBroadband tracker. It is the overall picture generated about the extent to which the site is accurate that matters. Labor’s survey released last week contributes to that overall picture and is therefore useful.

As I wrote in February:

“The interesting thing about this crowdsourced data that has come through regarding the MyBroadband site is not that it shows that the Government’s data is inaccurate. That’s to be expected; as Turnbull himself pointed out at the site’s launch press conference, the site uses estimates and not real-world measurements to give Australians a picture of broadband availability in their areas. By definition, it cannot yet deliver totally accurate data.

The actually interesting thing is that so far, almost all of the real-world data submitted is in one direction — worse than the estimates. The overwhelming trend seen from the many samples collected by Neate and her ad-hoc team of statisticians on social media is that real-world broadband speeds are significantly slower than telcos are estimating, with only a few outliers being better.

Of course, just as it’s possible to question the accuracy of the data collected by the Government’s MyBroadband site, it’s equally possible to question the data being collected by Neate and co. The blogger’s survey is hardly scientific, being based on voluntary contributions online. Nor do we know how qualified her assistants are to analyse that data. We’re not talking about a definitive study here. This information is very, very far from being verified and should not be treated as conclusive.

However, none of this invalidates the clear overall, high-level trend we’re seeing coming from the sample collected so far. Australia’s telecommunications industry has long been full of rumours about the poor quality of Telstra’s copper network, the degree to which it is affected by something as simple as rain, and the claim that many Australians currently on ADSL broadband aren’t achieving anywhere near the speeds they have been led to expect.

For the first time, we’re seeing this play out in reality here, with community-sourced, real-world data clashing directly with ‘official’ data from telcos and the Government. That clash has already appeared to reveal a significant gulf between the official version of the broadband truth in Australia and the real-world experience. We hope Minister Turnbull acknowledges this reality as the Government’s MyBroadband site starts collecting its own crowdsourced data.

And we wonder what impact these new data sources will have on the debate about the potential speeds achievable in a Fibre to the Node scenario for the Coalition’s Broadband Network project.”

Image credit: Labor

38 COMMENTS

  1. i used to live in morley up until november last year and i have absolutely no doubt that the actual average speeds are sub 5mb/s.
    my own adsl2 connection would often only connect at around the 3-4mb/s range.

    there are no shortage of PMG pits in the ground.
    i also know for a fact that the copper i was running off was directly laid in the ground, not in a conduit – it’s not unreasonable to assume the rest of the area is direct laid as well.

    thankfully i now live in an NBN area.

    • Wow on that chart Morley is at like 25mbs.. Is that even possible? o_O
      “thankfully i now live in an NBN area.”
      Rub it in! :)

  2. Don’t forget that the claims are for speeds “up-to” a theoretical maximum, which no-one gets, not even those with the xDSL equipment on their front lawn

    To put that in perspective, I live in a new estate on a RIM with a Telstra “top hat”. The top hat is no more than 200m from my phone socket (it’s in the same street, we are number 11, so 5 properties away), and I manage a sync speed of ~22Mbps.

    The best actual connection I can get however is 11Mbps, dropping to 3-4Mbps in after-school ‘peak hour’. MyBB suggests I can get “up-to” 24Mbps which is technically true, and therefore accurate for me, but not for anyone further away from the node. It’s misleading (but not false) to claim that the headline theoretical maximum is what one can expect on any xDSL system, but try telling that to anyone non-technical, and even if they do get it, the generally don’t understand what the difference is anyway.

    • The technicalities of the “up to 24mbps” are irrelevant here. The speeds on Turnbull’s website are meant to be real-world estimates, ignoring that theoretical top speed of ADSL2+.

      • Turnbull’s “real world” sounds great where everyone has magical internet speeds.
        Where do I join this real world :(

    • > To put that in perspective, I live in a new estate on a RIM with a Telstra “top hat”. The top hat is no more than 200m from my phone socket (it’s in the same street, we are number 11, so 5 properties away), and I manage a sync speed of ~22Mbps.

      So pretty close to the theoretical limit of ADSL2+ of 24Mbps. With an upgrade to VDSL you could be looking at close to 100Mbps.

      > The best actual connection I can get however is 11Mbps, dropping to 3-4Mbps in after-school ‘peak hour’.

      So there is congestion on your connection during peak periods. Have you identified where the congestion exists? Is it that Telstra have skimped on the speed of the fibre connected to the RIM, the congestion caused by your RSP, the sites you are visiting or …

      Since the issue is unlikely to be between you and the node, it is not a ‘last mile’ issue.

      • Any how is this ‘skimping’ likely to be fixed under FTTN?

        All he was guaranteed was 25meg (sans congestion) before 2016, and that has now been put by the way side…

        More of the same.

  3. A lesson in statistics.
    myBroadband = MEDIAN speed
    Your article “AVERAGE broadband speeds ….. universally below the data produced by the site”

    Example:
    A set of numbers, representing Mbps:
    1, 2, 2, 3, 5, 5, 5, 6, 6.
    Median = 5
    Average = 3.89

    In this case, average is almost 25% below the median.
    By extension, myBroadband’s number would be 5 Mbps, and average (what you refer to in the article) would be 3.89 Mbps. For the same numbers.

    Average is below the median because it CAN BE (it can also be above, in different instances).

    I’m no huge fan of myBroadband either, but atleast know what you’re talking about before you go into overdrive. Whether median is a good measure is another argument.

    • However, the median speeds claimed by myBroadband are unlikely to be correct.

      My area can only get ADSL1, which has a theoretical maximum of 8 Mbps. The median speed is said to be 8Mbps on myBroadband. This is highly unlikely.

    • hey mate,

      the article says “average” because that’s what Labor’s survey claimed to have calculated. I took statistics at university too.

      Renai

      • Renai,

        Where is the link to the original Labor image – it is hosted on your site, devoid of any citation?
        You provided no quote from Labor stating that it was “average”, the only use of that term was your own writing.

        My local ADSL2+ speed is 11.8Mbps median, my own speed is 2.1Mbps.. but n=1 survey says it isn’t even close, but myBroadband tells me that there are 221 other houses in my area… perhaps the majority has >= 11.8Mbps (because they have to in order for it to be a median).

        I took stats at uni, and use it every day as a scientist, but I don’t use misinterpretation of it to colour up an argument.

        • The image was sent through by email. Mate I honestly don’t know why the hell you’re attacking me. But I would remind you of our site comments policy:

          http://delimiter.com.au/comments-policy/

          “Firstly, as before, comments must be more or less ‘polite’, as measured by Australian social standards. This doesn’t mean you need to maintain the sort of conversation level you would use with your mother. It just basically means don’t be rude to other commenters. You may disagree with their opinions, but you should respect their right to hold them. This rule especially applies to the treatment of article authors, who deserve a significant amount of respect for putting their writing into the public domain.”

          • That’s mostly true, in a large normal distribution. However you’re assuming the dataset is normal, and not skewed. Do we know that, and have the numbers to prove that? I’m not the best person to state either way, but I’d suggest that someone with an intimate knowledge of the network topology in each area is better placed to.

          • You are assuming the data is skewed not only that skewed in one direction but not the other.

          • Good point AJ, a binomial skewness is a strange factor isn’t it. Neither median nor average really is a good measure of the population in that type of distribution are they! :)

    • I would think the Median speeds should be lower then the average, based on simple geometry.

      As an example, assume an adsl footprint of 5km
      4% – The number or homes within 1 km from exchange.
      12% – The number or homes from 1-2 km from exchange
      20% – The number or homes from 2-3 km from exchange
      28% – The number or homes from 3-4km from exchange
      36% – The number or homes from 4-5 km from exchange

      The average distance from the exchange is 3.53 km = SQRT (25/2)

  4. My understanding is that Turnball got his numbers from the telcos, who have every incentive to be overly optimistic about the quality of their connections. They don’t even have an incentive to quantify the connection speeds people are getting – they just label them “up to” x Mbps.

    That said, I don’t think Turnball is interested in the facts either – he seems more interested in spinning the facts to suit his agenda (which really shouldn’t surprise anyone, given that this is politics).

    • From playing around with the site, it seems more likely the speeds are based purely on distance from the exchange. My suburb has a couple of RIMs to supply a couple of areas that were developed in the last 10 or so years. Speeds of the houses supplied by these RIMs are in line with distance from the exchange, not what they would be getting from the RIM. So in there case the site predicts way lower speeds than they would get. Good that it is underpredicting in these cases, but bad in that it seems the data may not be based on real work speed data points by just exchange distance.

    • > My understanding is that Turnball got his numbers from the telcos

      Hopefully so. Internode and iiNet published a heat map for Sydney based on data from their DSLAMs back in 2007. This is the most accurate data source for the capability of the last mile.

      Labor’s numbers are most likely to be lower than actual performance. For example I could take a 100Mbps connection, limit the wireless access point to 802.11b, start a couple of heavy downloads and then run the speed test. The result could easily be as low as 2Mbps.

      • Sure, any number can be doctored, or the incorrect methodology applied in the calculation (eg actual download speed is heavily influenced by other apps using the same connection, the server connected to etc). But the general outcome is probably correct.

        At least in my case, the myBroadband website says the median speed is 7.42 Mbps, but I can only get a more or less stable ADSL2+ sync (not download speed) at 6 Mbps max. Which isn’t too far off (‘median’ means there should be a range of values on either side after all), but definitely not where we should be in 2014. However, I’ve heard of others with much worse connections than mine.

        • Yeah like mine, where myBroadband says 22.13Mbps and I get 4.3Mbps. There is a top hat in the area which is probably where they got the 22.13 from, but there is no way to connect to it. Existing connections are already connected to the exchange about 4km away and there is no way to get switched over.

  5. It’s going to be one lie after another, why do these headlines not reflect peoples’ lack of surprise ?

    • Im surprised that the general public and media are so surprised when they find out Malcolm Turnbull has lied.
      Turnbull has been unable to tell the truth on any aspect of Labors NBN or the expensive costly 2nd rate FTTN.
      My kingdom for somebody in the media to REALLY go after him full throttle and hold this compulsive liar accountable and to the truth.

  6. I sent a message to myBroadband from their contact link when the site went live. I recieved a reply probably a month or so ago now stating that in April they would be adding a speed test to the myBroadband site.

    I am waiting for this to happen so I can test all my friends and families connections. Not one person I have checked with syncs at the sites stated median speeds, let alone downloads at those speeds. This is of course not scientific proof but does seem to be the general consensus.

  7. Well I wouldn’t trust any of the sources. My Broadband is LNP’s play thing and this graph is Labors. Secondly their graph should be showing Mb/s or Mbps. Not that the average punter would notice but hey shows you it wasn’t exactly meant to be accurate and just a piece of political fud.

    The NBN has been reduced a political football, there is so much misinformation out there is no way for the average Joe to know what the truth is.

  8. I am beginning to wonder if the speedtest function will ever be built into the site. Although it might have been the original intention, now it is pretty clear that the real speeds will make a mockery of the published data.

    If they do put in a speedtest, will it be an accurate one? I can picture a speedtest function which mysteriously seems to report speeds 20% faster than those I get on legitimate speedtest sites.

    Although that might be just too clumbsy, even for the ham-fisted Turnbull propoganda machine. Perhaps they will use a real speedtest but give the results almost zero weighting.
    So we might have a listed speed of 25Mbps on the tracker, and after 1000 people do the speedtest and only get 4Mbps then the listed speed on the tracker is adjusted down to 24.9Mbps.

    The only guarantee we have is that any information listed on the site will place political utility as the highest priority. Completeness and accuracy of the information will be either a low priority or actively discouraged.

    • I’m not at home, so can’t really use it – but it looks like the site now has the speedtest built into it?

      • Plus it says it uses Ookla – that hopefully means it will give consistent results with other speedtests.

        • The speed test is not representative of the actual sync speed. these are different. Speede test will always be lower depending on the selected server to run the test from, the congestion in the ISPs network, as well as the whoevers server is used. I will consider myself one of the luckier BB users on ADSL syncing @18Mb/s (with my previous ISPI was syncing at 22Mb/s). NBN as wholasaler is/will be responsible for the sync speed; ISPs are “responsible” for the speed test performance (they decide on the contention ratios). People should not expect that their sync speed will be the speed at which they will be downloading all the, regardles whether it is FTTH o FTTN .
          This is not to say that I am against the NBN. I am passionate about it!!!

  9. The myBB site is also misleading when it states that HFC is available in your area, but I cant get HFC because either:
    A) Its not available in my street
    B) I live in a block of units and there are more people living in blocks of units than houses so (guesstimate) 60% of the ADDRESSES can get HFC but only 30% of the PEOPLE can.

  10. “In this sense, I would regard it [the survey] as quite suspect.”

    Given the narky nature of the Abbott government in perpetual opposition mode, wouldn’t they be quick to rubbish the survey data?

    If only they could.

  11. What is the cost difference between replacing the copper link from FTTN node to the premises, vs. the cost of the fibre terminal at the premises? I’m tipping they’re pretty close, i.e. equivalent cost for two radically different levels of capability. Subtract the rather significant costs of installing and maintaining the rather vandalism-prone street cabinets of an FTTN rollout (not to mention the cost to acquire the copper link from Telstra) and it looks like FTTP is the much cheaper option. I wonder just how expensive they think a bit of glass and a wall terminal really is?

  12. Pumpkin- high in fiber and low in calories, pumpkin packs
    a fortune of disease-fighting nutrients, including potassium,
    pantothenic acid, magnesium, and vitamins C and E. When our body p –
    H ( becomes more and more acidic it starts to set up defense mechanisms to
    keep the damaging acid from entering our vital organs.
    Blueberries Chock filled with anti-oxidants and also phytoflavinoids,
    Blueberries are also high in potassium as well as ascorbic
    acid.

    Feel free to visit my page are athletic greens any good

Comments are closed.