New World Order: IBM’s Qld Health disaster places project governance focus back on vendors

0

full opinion/analysis by Renai LeMay
9 August 2013
Image: Patrick H, Creative Commons

The disastrous, billion-dollar failure of Queensland Health’s project to replace its payroll systems delivers a stark warning to technology giants such as IBM that they must start to take a much stronger role in ensuring government IT projects they are involved with do not go disastrously off the rails. The alternative, as Big Blue found this week, is that vendors will increasingly find themselves locked out of government contracts entirely by vengeful politicians looking for scapegoats.

If you were strolling through Brisbane’s Botanic Gardens at around 2pm on Wednesday this week, you may have just felt as though you were somewhere close to paradise on Earth. It was a clear, light-filled afternoon, with the weather comfortably reaching the mid-20’s — ideal for the middle of winter. The Brisbane River was shining gloriously in the sun. Birds in the trees were singing, and all the verdant teeming plant life of the gardens was on full display. Food, drink and company were all easily available. In short, in that place, at that time, it would have been hard for anyone to feel that anything was truly wrong with the world. It’s a feeling that many residents of the Sunshine State frequently experience, as they rightly enjoy their wonderful natural environment and growing prosperity. But it was particularly strong on Wednesday.

However, just a few metres away inside the packed and often claustrophic chamber which is the city’s Parliament House, things were very far away indeed from paradise. The state’s fiery new Premier, Campbell Newman, had risen to his feet to address the House. In his hands was clutched a thick paper document, heavy with authority, and wroth was writ large upon his brow.

At the announcement of his topic, the report delivered this week by a State Government Commission of Inquiry into the Queensland Health payroll systems upgrade disaster (PDF), the Opposition Labor Party members — numbers diminished as they were, amounting to a paltry seven seats in a Parliament of 89 — immediate raised their voices in an uproar of vocal protest. “Listen to the Opposition!” thundered Newman in response. “They do not want to hear it. Sweep it under the carpet!”

It is common for politicians to take to their feet in parliaments right around Australia and to deliver stern lectures about the failings of their opponents. Sometimes other targets are also found: Special interest groups, criminals, rebellious elements of society, and of course, the media, which it is every politician’s earnest duty to admonish on a regular basis.

In this case Newman did cast down his displeasure on the previous Labor administration in Queensland, led by then-Premier Anna Bligh. But he also found time to lambast a more unusual target: One of the world’s largest and most venerable technology companies. A household brand which virtually everybody involved in the IT industry has worked with at some point. An organisation which every major government right around Australia depends on, on a daily basis, to get the very business of governing done. IBM.

“For its part,” Newman told the chamber in onerous tones about the payroll project, “it appears that IBM took the state of Queensland for a ride. I repeat what I said yesterday. The inquiry found: ‘The replacement of the Queensland Health payroll system must take a place in the front rank of failures in public administration in this country. It may be the worst.'”

“As a result of the Labor-IBM payroll deal and the findings of this Commission of Inquiry,” the clearly angry Premier went on, “I am announcing today that the State Government will not be entering into any new contracts with IBM until such time as IBM can demonstrate that its governance and contracting practices have been strengthened to deal with past misconduct and prevent future misconduct. I will expect IBM to deal with employees who have been adversely named in the Commission of Inquiry report, no matter where they are located in the IBM world.”

After Newman came Queensland ICT Minister Ian Walker, who noted that he was working on what he described as a “clear accountability framework” which would set out the roles and responsibilities of all stakeholders — including vendors like IBM — in future IT projects in the state. And then arose Health Minister Lawrence Springborg, who, like Newman, boldly and loudly attacked IBM’s credibility in the Parliament, even naming a specific IBM employee who had been key to the company’s contract in the Queensland Health payroll case. “Because of the steps of the Premier and what he has decisively done today, IBM will need to report on its efforts to change its corporate culture,” Springborg warned the company.

In a certain sense, the displeasure of the Queensland Government with IBM — which extended to a formal parliamentary censure motion against the company — can be seen as meaning little to Big Blue. After all, it’s nothing that IBM hasn’t seen before many times.

Unlike most of the world’s modern day technology giants, IBM is not a short-lived company. It was founded more than 100 years ago in 1911. Since then, it has endured through a clutch of serious wars involving its home country of the US, massive and constant market and technological revolutions, immeasurable societal change and a constant line of acquisitions and sell-offs that makes the history of almost any other company look like the ephemeral flap of a butterfly’s wings.

We may recall, after all, that IBM controversially supplied equipment to both the US and German governments during World War II. IBM equipment was used in the early space flights, including the Apollo missions which landed men on the moon for the first time. IBM has made everything from time clocks and basic tabulating machines to modern-day PCs, supercomputers and even quasi-artificial intelligence agents in its time; and there is reason to suggest that this is one company that has been able to continually evolve itself and change over its history. With a corporate memory that deep, one may wonder whether anyone at IBM this week even blinked in the face of Campbell Newman’s fury.

Then too, IBM has been banned by bigger governments than Queensland before.

In April 2008, for example, the entire US Government blocked new contracts with IBM, after an investigation by the Environmental Protection Agency found IBM had violated procurement laws while negotiating a contract with the agency. The case had remarkable similarities with the situation with Queensland Health, in that in the EPA case, several IBM employees involved in the procurement process reportedly obtained information from EPA employees which other competing firms did not have, that would have given IBM an advantage during the procurement process. However, the ban lasted only a little over a week, after a settlement was made between the two sides. In practical terms, one suspects it would have proven impossible to enforce in any case.

And yet, Newman’s move to ban IBM from new contracts with the Queensland Government is still significant.

Right now, when it comes to major enterprise IT projects and IT services contracts in Australia, there are only so many vendors which the largest government departments and companies depend on to get the job done. If you have a project worth in the tens of millions, there are quite a few options. But when it comes to a project or contract in the hundreds of millions, the list of suppliers capable of meeting your needs quickly narrows to just a handful. And it is precisely in these areas that IBM makes much of its $4.5 billion in Australian revenue annually.

In 2006, for example, when a report into the wrongful detention of Cornelia Rau and Vivian Alvarez led the then-Department of Immigration and Multicultural Affairs into a wholesale revamp of its IT record-keeping systems, it was a consortium led by IBM who picked up a $495 million deal to oversee the so-called ‘Systems for People’ project. IBM has contracts worth hundreds of millions of dollars with the Department of Defence relating to its hardware and software, and is a key contender in the department’s upcoming $500 million centralised processing contract. It is the key IT outsourcer at top-tier bank Westpac, where IBM has won several billion dollars worth of work over the past decade, and it’s a similar case at the National Australia Bank. In March 2011, IBM picked up a $200 million deal with the National Broadband Network Company to deliver key IT systems, and it’s also a key contractor to Telstra.

Of course, IBM has competitors. In Australia, it regularly fights for these massive contracts with rivals like HP, Fujitsu, CSC, Unisys, Lockheed Martin, Accenture and others. But it remains a fact that IBM always wins its fair share. In an industry constrained by human resources (an issue particularly pertinent for the public sector, which usually refuses to use offshore labor), it’s hard for any one company to lock up enough talent to be conducting too many major IT projects and contracts simultaneously. What this means is that Australia’s major IT services companies usually each hold a decent part of the pie. At a certain level of enterprise IT purchasing, it is usually impossible to avoid dealing with IBM eventually — and that’s not saying anything about its hardware or software businesses.

For this reason, it doesn’t seem likely that other states, the Federal Government or private sector companies will follow Queensland and block IBM from winning new contracts. The practicalities of such a ban are more trouble than they are worth. In addition, it doesn’t even seem likely that Queensland’s ban will last that long. IBM will conduct a small internal review to satisfy the politicians, and after a few months, will likely be let back on the state’s roster.

However, Queensland’s move to block IBM will have two important impacts on the company’s operations in Australia. This is not the sort of problem that any company walks away from unscathed.

Firstly, the revelations contained in the Commission of Inquiry report will have the immediate effect that any procurement process involving IBM will be tightened up, and the company’s participation placed under intense scrutiny. The reason for this is the evidence presented in the report that IBM behaved inappropriately throughout the procurement process for the Queensland Health payroll system, and should not have won the contract in the first place

The most obvious way IBM was out of line during the procurement process this was to take advice from government staff with respect to its bid that the company, the Commission of Inquiry found, must have been aware was clearly inappropriate. This, in and of itself, is enough to make anyone negotiating with IBM in terms of a major contractual engagement very cautious in future negotiations. However, the far more troubling point made in the report, to those of a technical mind, is that IBM actually proposed a technology solution which was not appropriate for the task and which had not been tested properly. The report state:

“IBM’s proposed use of Workbrain, though offering accelerated roll out and reduced cost, was risky because it was untried. That was the view of the Panel members. [Government staffer Terry Burns] urged the rewarding of innovation and his argument was accepted but with the qualification that the reference sites had to provide proof that the proposal worked in practice. That proof was not forthcoming. Logically, then, the situation should have reverted to the original. There should have been a third appraisal resulting in the reinstatement of the Accenture bid as the better one because it was less risky. But rationality and sense gave way to haste and muddle, and the State committed itself to an untried solution.”

It is one thing to accept confidential information during a procurement process and use that information to lowball a bid, undercutting competitors and presenting an unreasonable view of the finances of a project implementation. Many people would see that as just good intelligence and salesmanship, and this kind of activity is certainly far from unusual, even in the public sector. I’ve seen a number of audits into government ICT procurement processes which have flagged similar inappropriate behaviour, if not quite at this level.

But to propose a technical solution which wasn’t tested, and then to neglect to provide evidence of its fit for purpose when asked, is a much greater crime and implies incompetence, instead of merely smart commercial behaviour. By taking this path, IBM endangered the whole Queensland Health payroll systems upgrade; and it should be obvious by now how catastrophically far off the beaten track the project ended up. Workbrain was a bad solution for IBM to bet on, and one suspects the company could have come up with a much better one, given the depth of its global experience.

At the moment, there are a large number of similar (although usually smaller) payroll systems upgrades going on around Australia to Queensland Health’s project. Implementing modern payroll, human resources, CRM and financial systems is one of the greatest challenges that Australia’s many departments and agencies are facing right now. IBM’s poor technical approach to the Queensland Health upgrade will raise many questions when it bids for similar work in other jurisdictions over the next several years. The company will very likely find technical evaluations of its proposals very tough indeed, from now on. Nobody will want to make the same mistake Queensland Health did — picking the wrong solution, an untried solution, because it was dramatically cheaper.

The second impact which the Commission of Inquiry report and Queensland’s ban on IBM will have will not be constrained purely to IBM’s operations in Australia. It will inevitably affect every other major IT services supplier operating in Australia as well.

A series of audit and ombudsman’s reports published over the past several years have conclusively shown that as a whole, our country’s state governments are comprehensively unable to deliver major ICT projects on budget or on schedule, and that they are also pretty bad at delivering ICT services to departments and agencies.

In November 2011, for example, Victoria’s Ombudsman handed down one of the most damning assessments of public sector IT project governance in Australia’s history, noting total cost over-runs of $1.44 billion, extensive delays and a general failure to actually deliver on stated aims in 10 major IT projects carried out by the state over the past half-decade. Similarly, in Queensland, a recent ICT audit found ninety percent of the Queensland Government’s ICT systems are outdated and would require replacement within five years at a total cost of $7.4 billion, as the state continues to grapple with the catastrophic outcome of years of “chronic underfunding” into its dilapidated ICT infrastructure. At the heart of almost all of these projects and ICT service delivery operations is a major supplier like IBM, which has won a substantial contract at some stage to deliver the projects or services.

IBM successfully exited the fraught legal situation with the Queensland Government over the Queensland Health payroll issue. But Newman’s decision to ban the company from further engagements with the state takes the situation to a new level: A level which will see Queensland insist, in future engagements on these kinds of major ICT projects, that the prime contractor involved takes much more responsibility for project governance. With his ban, Newman is saying to IBM that, even though Queensland cannot hold IBM legally or financially responsible for the Health payroll mess, it is holding it morally responsible for its lack of action in ensuring proper project governance of the upgrade to start with. And not just for this project: But for all similar projects in future.

And if you read what the Commission of Inquiry says about IBM in its report, Newman’s stance is more than justified. Right from the very start of the project, the Commission of Inquiry found, IBM neglected its duties with regard to the project. The report states:

“IBM seemed content to have defined scope in a way which it must have known could not result in a functional payroll system and to rely, whenever challenged, on the text of the [Queensland Health Implementation of Continuity] Scope Definition. Its task was to gather requirements. It did so incompletely, relying on its obligation to do so stopping short of being responsible for eliciting such requirements from its customer. The State, on the other hand, was recalcitrant in adopting
a less than diligent approach to ascertaining and communicating its business requirements to IBM. It regarded IBM as responsible for eliciting requirements from it, and considered instead IBM’s obligation to 113 deliver the project to be for a “stable, supported and supportable” system. In short, the State assumed it had contracted for, and would get, a system which paid staff accurately, perhaps with some manual intervention, but, largely, to do so in an automated way. IBM considered its obligation, subject to some matters set out below, to be to deliver the system which would follow (and no matter what it looked like as a result) from a strict application of the QHIC Scope Definition.”

This extraordinary statement reveals much about IBM’s project management methodology with respect to the Queensland Health payroll project. It appears that IBM was determined to stick to the letter of its agreement with the government — despite the fact that it knew that its project scope definition was deficient. The short-sightedness of that approach, given the end results (a billion dollar blowout and public relations disaster for the Labor administration which contributed to its electoral downfall, as well as the human cost to the thousands of health staff who went unpaid), is striking.

There are also other indications that IBM knew the project was failing but did not take action. The report states:

“I have mentioned above that both parties knew before completion of the Project that the system would, without some major reconsideration of it, fail to pay staff, or do so in a seriously inaccurate way. Neither party followed basic project management principles by insisting that the Project be reset or by taking other effective measures to rectify problems which remained with the system until after Go Live … Any rational assessment would have deferred the Go Live, with some very determined effort to reset the Project and make some alternative arrangements for the ongoing support of the incumbent system, LATTICE.”

And several other key paragraphs: “IBM was more than a software developer. It was a prime contractor, and one which had held itself out in its [Invitation to Tender] response as having experience with systems of this kind. It was both designing and building the system. These factors show IBM to have had an obligation to be more than merely a passive recipient of the customer’s instructions.”

“The system was inadequately scoped and neither party took any effective measures to stabilise it. Both parties ignored all the warning signs of a project in serious distress. Rather than reset the Project or take decisive steps to put it on a stable course, they altered or lowered the thresholds which had been put in place to protect against the very thing which eventuated: a system of poor quality which was not ready to Go Live. Doing so meant that the system, when it went live, would not be functional and constituted a clear breach of the most fundamental principles of project management.”

None of this is to say that the Government did a good job on the payroll project either. A string of reports into it have largely found the Government responsible for the issue, rather than pointing the finger at IBM. The Commission of Inquiry report is the first examination of the Queensland Health payroll issue which has leveled serious allegations at IBM.

However, what we see from the report is a very clear indication that, for its part, IBM did not follow correct project management procedures in the delivery of the Queensland Health payroll replacement system. Newman’s statement in Parliament, and the ban on further contracts with IBM, is a clear signal to IBM that the state government will not tolerate the vendor taking a “passive” approach to project management in future projects of this kind. The Newman administration will start expecting a lot more from all its vendors in major ICT projects in future; as ICT Minister Walker’s development of a new accountability framework also demonstrates.

Currently, the way most large ICT projects are run by government departments is that an internal departmental project office maintains overarching control, with outsourced partners doing most of the actual work but not controlling the process. Newman is signaling that this traditional style of project delivery will change under his watch; partners such as IBM will be expected to take a much more active dose of responsibility for the projects they’re delivering. If the project is failing, as the Queensland Health payroll project broadly failed, then vendors like IBM will be expected to raise their voices strenuously about that situation before it happens. In fact, one would expect that they should not allow such disasters to happen, as IBM clearly did in the Queensland Health situation.

It is also likely that Newman’s approach to this issue will percolate through to other state and federal government departments.

The failure of the Queensland Health payroll upgrade is a once in a decade-level event, akin to the 2005 Integrated Cargo System ICT catastrophe at Customs which resulted in wharves piling up with a backlog of cargo which Customs’ new platform, delivered by EDS, was unable to process in a timely manner. As were the audits into the ICS disaster, Queensland’s Commission of Inquiry report into the Queensland Health issue will be widely read and extensively discussed by chief information officers and departmental secretaries right around Australia over the next six months. The lessons contained in its pages and Newman’s highly public outright demand that IBM take responsibility for its actions, will not go unnoticed by bureaucrats seeking to implement similar projects. Nobody will want to see that situation repeated.

What I expect to see over the next several years in Australia’s public sector particularly is a much stronger focus on holding vendors to account for the projects they work on. The customer must always take prime responsibility for project failures, but increasingly it seems obvious that we’ll see vendors taking more heat as well. The Queensland Health example provides a clear lesson in what happens when vendors are allowed to escape taking responsibility for major projects.

If you look at the situation from this point of view, the Queensland Health debacle need not be viewed as a complete negative. It’s been a while since Australia had such a highly visible IT project disaster, and the good thing about this one is that it’s been comprehensively documented from every angle.

For the next year, I’m sure that all of IBM’s rivals will out in the market trumpeting its failure to every customer that will listen, and I’m sure that IBM will find it difficult to win major new payroll upgrade contracts in the short term. In addition, there is absolutely no doubt that the Queensland Health example will see all procurements involving bids from the company tightened up, to ensure the same level of impropriety doesn’t happen again.

However, what is perhaps more important here is that Australia’s government and corporate sectors take heed and incorporate the lessons from the Queensland Health disaster into their own procurement and project management practices. The lessons to be learned from this situation can and will be applied not just to IBM, but to every other vendor involved in delivering major ICT projects and services. IBM is far from the first vendor, and it won’t be the last, to have behaved improperly during a procurement process and to have failed to adhere to basic project governance practices. But the rules of the game have changed. It will no longer be OK for vendors to take a hands-off role in project governance for these kinds of major projects. The danger of such an approach, as we’ve seen in the Queensland Health example, is that the vendor themselves will become the target of powerful political figures’ fury.

As for IBM itself? The old warhorse of the enterprise IT industry will quickly bounce back. As long as it makes a decent internal effort to reform some of the less competent areas of its business and gets back on the right footing with the Queensland Government, it will be back in safe territory very soon. It will never quite live down the ignominy of being right at the heart of one of Australia’s worst ever corporate IT disasters. Nobody will forget how IBM sat around and watched Rome burn at Queensland Health, and many people will be watching the company closely in future engagements to ensure it doesn’t let a similar situation happen again. But then that’s the cost of being in the game to start with. Most of IBM’s projects go more or less according to schedule. But occasionally the devil snatches one back: And it’s those that we will never forget.