news Telstra has confirmed that the management console for its corporate cloud platform went offline for some of its customers for two days last week, in the second demonstration in less than a year that the company’s cloud computing environment may not yet be as stable as the company would like customers to believe.
Delimiter received information last week to the effect that the telco’s Cloud Services Management Console had been intermittently experiencing major outages since the morning of 6 January. The issue did not cause a loss of service for the virtual machines and other cloud infrastructure which Telstra runs for customers in its datacentres, but did block access to management utilities and details about those machines, as well as the ability to purchase new or redistribute existing services.
Delimiter was told that the issue was serious enough that Telstra’s senior executive team, including the company’s chief executive David Thodey, was being updated regularly on the issue. Telstra is believed to have called in external assistance from at least one of its vendor suppliers on the issue, believed.
Telstra issued the following statement on the issue: “There were no connectivity issues to virtual servers or internet connections, but some Cloud customers may have had difficulty accessing the Cloud Services management console to view or modify services. The matter was identified on 6 Jan and rectified yesterday (8 Jan) evening. Overnight monitoring and subsequent checks have confirmed functionality has been fully restored. The incident has not affected our 99.9 per cent platform uptime commitment as customers could still access cloud environments throughout this time.”
The issue represents the second time in less than a year that Telstra’s cloud platform has suffered a serious outage lasting for at least a day. In April last year the telco confirmed that it suffered a major outage in its high-end corporate cloud computing platform that left a number of its most high-profile customers without some of their services for a period as long as 24 hours.
The news had the potential to knock the facility’s uptime capability almost out of the top high-availability tables for IT infrastructure. Under commonly used guidelines, technology services are classified by the amount of downtime they suffer per year. If substantial parts of Telstra’s cloud computing infrastructure was offline for the majority of one day last week, the company’s cloud platform may no longer be able to be classed as having uptime above 99.9 percent (‘three nines’), as this would entail suffering downtime limited to 8.76 hours per year.
It is common for enterprise cloud computing platforms to enjoy uptime higher than this figure. For example, cloud computing vendor Salesforce.com states on its website that its Force.com platform has had a proven 99.9+ uptime “for years”. The company publishes uptime statistics on its site.
The news comes as Telstra continues to push the case that its cloud computing platform reflects a sizable revenue opportunity for it going forward. In mid-2011 the company stated that it would spend $800 million in the space to develop its infrastructure and target new customers.
At the time, the telco said the $800 million spend will go on a range of areas over the succeeding five years, but would especially be focused on the construction of a new Melbourne datacentre, slated to go live in 2013, which would bolster Telstra’s hosting capability by more than 40 percent, with an extra 2000 square metres of space.
The money was also to go on modernising existing Telstra datacentres, expanding the range of enterprise applications the company provides, building a new integrated online account management portal, increasing the automation of utility computing services, and enhancing the capabilities of T-Suite. The company’s key partners for the next phase of its strategy will be Cisco on the hardware front, VMware for virtualization, Accenture for the company’s systems integration skills and Microsoft for software.
In December last year, Telstra announced it would construct four new datacentres to meet demand.
This is a very serious issue for Telstra. The company is talking about three nines availability for its corporate cloud platform (99.9 percent uptime), but clearly, no matter what it says publicly, it is having issues keeping to that benchmark — and three nines is actually not that impressive in terms of uptime. This is pretty much expected, given the relative newness of Telstra’s cloud infrastructure compared to more established platforms, but I’m sure the company’s major customers — including giants like VISY — are not happy with these outages, given the amount they are no doubt paying Telstra for the privilege of using its dynamic infrastructure. If you can say one thing about Telstra, it’s that the company never sells anything cheaply. The company’s ability to rectify faults in a timely manner — instead of taking days — also appears to be suffering. Does Telstra have a lack of IaaS troubleshooting expertise in its ranks?