It is late in the month of October in the year 2008, and Sydney has just begun the long wind-down as it prepares for its traditionally glorious summer, with all the pleasures that implies. The Christmas party is season is in full swing — muted slightly by the sharp budget cutting that has taken place with the global financial season — and silly season fever has taken hold of much of Australia’s technology sector, as it does this time of year, every year.
But despite all the frivolity, Optus managing director of products and delivery Andrew Buay is not having fun.
Buay’s long-running problems started in July that year. A cable cut by non-Optus contractors took down the entire state of Queensland in one hit, leaving hundreds of thousands of customers stranded. Then a series of glitches in August hit the company’s 3G network. The launch of the iPhone 3G on Optus’ network in July went fantastically well, but took its own toll on the telco’s infrastructure, flooding its mobile base stations with hungry smartphones at a time when it could ill accommodate them.
Throughout this whole process the press was taking note, making a little notch on its collective belt every time Optus suffered another glitch, with many keenly awaiting what they saw as the endgame for the troubled SingTel subsidiary: Total network meltdown, leading to mass customer exodus.
Yet at the time Buay himself appeared to remain calm.
The carrier’s ongoing spate of problems with its 3G mobile network were “quite normal”, he insisted in an interview with ZDNet.com.au. “I suppose mainly because we have been in the limelight, every little thing that does happen gets a lot more visibility, but you know it does happen with every operator and network in various sizes and frequencies.”
How prophetic those words must seem to VHA chief executive Nigel Dews today.
This week the experienced telco executive — veteran of a thousand network storms — was forced into taking that most unhappy of steps in a CEO’s public life: The general apology. With Vodafone’s call centre crushed under an onslaught of complaints about outages in its troubled mobile network and the internet teeming with thousands of angry customers, Dews took to the company’s blog to appologise for the telco’s ongoing problems.
“I would like to apologise for recent intermittent network issues that have impacted some of our Vodafone customers and how we have kept you updated,” he wrote.
“Having customers who are happy with their service and their network experience is central to us, but unfortunately in recent weeks, some customers have had a disappointing and frustrating experience which I am very sorry for. Looking at your comments on various blogs including here on our own, it’s clear we could have done a better job at keeping you across what’s been happening.”
Like the problems suffered by Optus throughout the second half of 2008, Vodafone’s problems over the past few months appear to have come on the company one by one, like a slowly building avalanche that suddenly crashed down on its head. On October 20 one of the first posts about the issue hit the forums of broadband information site Whirlpool, with one customer from Adelaide noting problems such as extremely slow internet browsing, time-outs, SMS failures and low signal strength had started apearing “about a month ago”.
After just a week it appeared the general public had caught wind, and Whirlpool’s Vodafone forum started to fill to bursting point. That first thread now has 54 pages of comments and others can be found both within Whirlpool and on other forums across the internet.
The Twitter hashtag #badoptus had long acted as a central point of complaints about the SingTel subsidiary’s network performance, but now a new sounding board was born — #vodafail. By early December, a whole website had been set up to chronicle the complaints — Vodafail.com — to aggregate hundreds of customer stories.
If you believe some of the angry rants, as a brand Vodafone could be close to the end (the Vodafone brand is operated by VHA in Australia).
“The amount of work that’s required to fix this mess is going to be almost impossible and I’ll take my hat off to Vodafone if they can fix the problem,” wrote one customer, in a post which is one of Vodafail’s highest-rated rants. “Word of mouth is how they got so popular but it could possibly be the end of them.” “I think Nigel Dews should resign. Your sorry is too little to late,” wrote another. “Your pathetic excuse that only a few people are suffering is just that, pathetic.”
And yet, when you delve beneath the surface of what’s actually going on within Vodafone’s troubled corporate walls right now, the story is a little more complicated than the telco’s critics would have their audience believe.
For starters, IDC telecommunications analyst Mark Novosel points out customers’ experience of the network will depend on “many different factors” — where they live, how congested each mobile tower is, and sometimes even handset choice. Characterising the Vodafone mobile network as a single piece of infrastructure with a fundamental flaw is understating its complexity — like any mobile phone network, it is a linked system of interconnecting networks spread out across the nation.
But remarkably, also, Vodafone’s situation today shares many similarities with the problems being experienced by Optus throughout 2008.
Both telcos suffered widespread problems in a certain state relating to a fibre cable outage (Western Australia for Vodafone, Queensland for Optus). Both have suffered from software problems in the network that affected stability as a whole. And both had a series of glitches in capital cities, where data-hungry early technology adopters are likely to complain the loudest.
Macro-level problems also affected both telcos.
Like Optus in 2008, Vodafone is right now constantly throwing capacity at a mobile phone network which is experiencing exponential growth in data traffic. In 2008, Optus was dealing with an influx of iPhone traffic onto its network, with the greedy Apple device making a mockery of the consumption habits of previous-generation smartphones such as the more efficient BlackBerry and even Nokia’s Symbian platform.
Today Vodafone is facing a similar situation. In 2010 Australia’s carriers are facing not just the iPhone connecting to their network, but a raft of similar handsets based on Google’s Android platform. Even low-end phones that cost less than $200 are now increasingly based on Android and have full web-browsing and multimedia capabilities.
In the understatement of the year, the Australian Communications and Media Authority wrote in a December report this year that the take-up of mobile broadband services in Australia (including smartphones and USB modems) had “grown significantly over the past two years”.
In December 2008 — as Optus was having its own well-publicised problems — there were some 1.3 million mobile broadband users in Australia, according to the Australian Bureau of Statistics. A year and a half later, another 2.1 million connections had gone live — with many of those on Vodafone.
Over the past year, all three of Australia’s mobile telcos — Vodafone included — have made public statements about how much they’re investing in capacity to their mobile towers to meet the demand. It’s one of the chief benefits the trio are looking forward to from Labor’s flagship National Broadband Network project — they’ll be able to run fibre easily to any mobile tower when it’s complete.
Staying ahead of this capacity game is understood to be one of the complicating factors behind Vodafone’s outages. When one system goes down, it delays the constant upgrade cycle, and so you’re already behind even when you’ve fixed the issue — like trying to put out a fire with a constantly leaking bucket.
Novosel agrees Vodafone’s issues are a lot like the ones suffered by Optus in 2008. And, he points out, Optus has been able to largely resolve most of its problems since that time, through “a significant amount of investment”. Vodafone, too, is known to have been investing heavily in its network for some time.
The SingTel subsidiary isn’t completely out of the woods — as an outage several weeks ago demonstrates — but Novosel says the network has improved greatly. Indeed, IDC published a report in June showing Optus’ 3G mobile broadband offering was only four percent behind Telstra’s Next G across a range of criteria, although Telstra’s network was, on average, much faster.
However, Novosel says Vodafone’s situation, when it comes down to it, more complex than that of Optus was back in 2008.
Vodafone isn’t just improving its network right now, he points out — it’s actually simultaneously integrating the network of its acquisition Hutchison (which operated the ‘3’ brand), adding in 850Mhz network assets to help its existing network, and building using the 900MHz spectrum in regional areas. Hundreds of millions of dollars are being ploughed into such initiatives as well as general network upgrades.
“It would appear that it’s more complicated,” the analyst says.
Then, too, the company’s network is about to hit crunch time. The Christmas and New Year period are always the busiest time of the year for Australia’s telcos, with millions of text messages and calls placed to loved ones as the entire nation takes a break. Vodafone’s network will be tested to the absolute limit and back again over the next week — as will those of Telstra and Optus.
But one gets the feeling about Vodafone right now that the company has an “all hands on deck” attitude about it as it swings into the New Year. The image you get talking to company insiders is one of a hive of activity, especially on the technology front. And it’s reinforced by the company’s social media presence, which has been frantically responding to outage queries as fast as they can type this week.
And certainly Vodafone has already fixed its biggest problem: A lack of communication.
“Our message to all telcos is this: network problems will occur from time to time, and customers understand that,” said Australian Communications Consumer Action Network chief executive Teresa Corbin this week. “But if you’re in the communications business, you have to let people know what’s going on or you will risk losing your customers for good.”
It’s probably safe to say that Vodafone — and Optus before it — has now gotten the message.
Image credit: Vodafone