Making IT work

Like the nation’s roads and bridges, most state Information Technology systems need costly makeovers at a time when states’ can ill afford them — presenting a quandary for those aiming to make state government more efficient for the long haul.

Some more than 30 years old and written in outdated computer code, so-called legacy computer networks are slow, cumbersome, prone to crash and incompatible from one agency to the next. 

Pouring money into large-scale computer and data processing projects can reap huge gains in efficiency, accuracy and public opinion. Linking wide-ranging state agency systems, for example, can allow state governments to more easily share data, conduct interagency projects, move state employees from one agency to another and provide better services to the public.

But recent experiences in California, Indiana, Texas, Virginia and Wisconsin indicate that when these projects fail, or even slide off course, they can cost far more than the proposed overhauls were projected to save.

  FIVE KEYS TO IT CONTRACTING SUCCESSES

With 85 percent of all outsourced IT projects resulting in failures at some point in the process, state officials and their vendors rarely see eye-to-eye on everything. But based on experience, most agree states can take the following steps to ensure smoother sailing.

1. Involve all vested agency officials from planning stages through project completion and testing.
2. Ensure that vendors and state managers fully understand project parameters and scope before bidding begins.
3. When possible, conduct a two-stage bidding process in which finalists submit a prototype design before the winning vendor is selected.
4. Form a strong vendor relationship that balances accountability with flexibility.
5. Build a strong state management team to oversee projects through final implementation.

An estimated 85 percent of government IT projects fail to come in on-time, on-budget or both. “The IT project road is littered with failure, cost-overrun, mission creep, bad contractors, systems that don’t deliver, failed relationships between the parties,” ticks off Dianne Lancaster, chief purchasing officer in Oregon and president of the National Association of State Purchasing Officers (NASPO).

Still, in this current fiscal climate, states have little choice but to seek the savings that can come from contracting with private companies to achieve major IT transformations. To minimize risk in these high-stakes projects, some states are using new procurement procedures aimed at holding vendors accountable and getting vested state agencies involved from the beginning. California has created an entirely new procurement system to ensure that both the state and vendors bidding on the state’s lucrative contracts know what they are getting into before they sign the contract.

A look at a few recent failures of mammoth IT projects demonstrates why the precautions are needed:

• Texas suspended an $863 million government-wide data consolidation project with IBM in late 2008 after it suffered one too many security lapses and backup failures.
• Indiana in October 2009 canceled a $1.3 billion contract with IBM to transform application processing and record-keeping for its family services department.
• Virginia  — a year behind on a five-year, .4-billion IT contract with Northrup Grumman to create a new multi-agency IT infrastructure — has suffered repeated network interruptions. 
• California’s $170 million project to update its payroll system recently went off course and another project aimed at linking the states courts ballooned from $240 million to more than $1 billion. 
• Wisconsin suffered significant difficulties and delays with six large IT contracts before it revamped its procedures three years ago.

Each of these states experienced its own unique set of problems, but in general, major state IT contracts are notoriously difficult to pull off. That’s partly because state governments don’t operate the same way as private companies, and most state managers aren’t trained to direct major IT projects.

When consolidating network services across agencies, for example, the state and contractors have to reconcile the IT needs of organizations with very different missions and scales.

“You take a business like Procter & Gamble — they are a retail giant, but you know what they are doing,” said Doug Robinson, executive director of the National Association for State Chief Information Officers (NASCIO). State functions, meanwhile, are as varied as issuing licenses and vehicle registrations at the Department of Motor Vehicles to conducting long-term scientific research at the Department of Fish & Wildlife — and the price for interruptions of service are extremely high, not only in dollar terms, but also in public opinion.

Making these projects even more difficult, state personnel often lack the background or vocabulary in IT to articulate what they want. “We as a state might know we want the system to accomplish these business tasks, but we can’t tell you how to write code to do that,” said Erika Li, technology analyst with California’s Legislative Analyst Office. “And maybe we are not doing business the best way to begin with, but need to change that first and then automate it.”

So how can states avoid costly contracting errors and ensure the highest level of success?

To outsource — or not

The first pivotal decision is whether to contract with a private IT company or make the changes using off-the-shelf technology and a state’s own technical staff. Although most big state technology upgrades are outsourced, not every project is best handled that way — even when the price appears to be right.

Last year, for example, Indiana’s Republican governor Mitch Daniels conceded that the state’s .34-billion contract with IBM for processing family services applications “just didn’t work.” After numerous errors, lost files and long wait times, Indiana cancelled the contract.  IBM blamed the failure on a 33 percent increase in the number of people applying for social services that wasn’t accounted for in its contract, according to press reports.

Outspoken critic, Republican state senator Vaneta Becker, said the state would have been better off had it simply upgraded the technology used by current case workers. “This system cost the state a whole lot more money than had they just given each employee his or her own computer and phone line,” she said.

But when outsourcing does make sense, state officials agree the most important factor for success is forging a strong relationship with the vendor that balances accountability and flexibility.

Lessons learned in Virginia

In Virginia, what first appeared to be a balanced relationship with Northrup Grumman went badly wrong.

Launched in 2005, the government-wide IT project aimed to consolidate data processing across all agencies.  Upon signing the contract, the commonwealth touted its vendor partnership as one in which both sides would share in the success and take responsibility for problems.

A recent analysis by the Joint Legislative Audit Review Committee (JLARC) found, however, that “the largest single reason for delay (in implementing the network) appears to be inadequate planning by Northrup Grumman,” which failed to account for “hundreds of sites and thousands of assets” during due diligence in the beginning phase of the project.

Northrup Grumman, for its part, took issue with Virginia’s complaint. “No one is clairvoyant,” said Vice President Cheryl Janey, who pointed out the state had some 90 agencies with 2,000 locations, and 42,000 personal computers. “It’s not possible in a finite period of time to count all of the assets and do a complete due diligence. As soon as you enter into a contract, you know you will find things you didn’t expect.”

Grumman representatives said they discovered that Virginia had 10 times the storage capacity it said it had — translating to $40 million a year more to maintain it — a nearly 17 percent increase in the $240 million-a-year maintenance contract.

According to the legislative report, the state made matters worse by failing to impose contractual penalties on Northrup Grumman for falling behind schedule as a result of those early miscalculations. “There was some effort to go easy on Northrup Grumman in the first couple of years in this spirit of ‘partnership,'” JLARC director Hal Greer said. “When it’s not clear who is responsible, you can get into really dangerous waters.”

While a company’s reputation and money is at stake in these contracts, the state is paying millions of dollars for the work and potentially jeopardizing its entire computer network, according to NASPO’s Lancaster. “A partnership is defined as something where both parties have something at stake,” Lancaster said. “Well, where is the partnership here if you don’t have as much skin in the game as I do?”

But contracts that require “unlimited liability” or “performance bonds” are non-starters for most companies, said Michael Kerr, senior director for state and local government for the IT trade organization TechAmerica. Similarly, penalties that exceed the price of the contract can cause some vendors to balk or add a premium to cover their risk, Kerr said.

Still, some states include damage clauses that amount to more than twice the price of the contract, arguing that when a contract fails, rehiring outsourced workers and rebidding the project result in much higher costs. Others attach payments to completion of project milestones and stipulate that the team that presents a proposal is the same one that implements it.

Early agency buy-in

Even with an airtight contract, however, it’s not enough to sign the checks and watch the networks get wired. State IT officials say active contract management is critical.

Analysts who reviewed Virginia’s contract failures with Northrup Grumman say part of the problem was that affected agencies were not brought in during the early stages of the project to help coordinate planning and inventory.

Texas had even bigger problems in managing its contract with IBM. According to a legislative audit, some agencies actively impeded progress by refusing to share their most knowledgeable employees with IBM. After multiple problems, including lost data and security breaches, the state suspended the project to regroup.

One of its key changes was to create a new agency executive council with high-ranking officials from key agencies to include in the planning process. “What we have come to realize is you need to tailor the initiative to the specific needs of each agency,” said the state’s new director of the Department of Information Resources Karen Robinson.

Wisconsin experienced similar problems. After multiple IT project failures, the state’s new Chief Information Officer Oskar Anderson put a government-wide computing project on hold in April 2008 while he retooled contract management procedures.

Anderson required agencies to designate a manager for each project and insisted that an outside review board issue monthly status reports for department members with a stake in the project. Finally, Anderson established a technical review committee to ensure all state projects met established standards. Since then, Wisconsin has not had a single project failure. “That tells us we are either incredibly lucky or we have done something to improve our process,” he said, “and I’m not that big a believer in luck.”

A California ‘bake-off’

California, meanwhile, has completely overhauled its contracting system for large IT projects. Called multi-stage contracting, the process works like this: First, the state solicits vendors and gives a certain number of qualified finalists a chunk of money up-front to develop a prototype. Then, a second stage of bidding — called the “bake-off” — allows procurement officials to evaluate the prototypes and choose a winner.

While multi-stage contracting requires more time and money up-front, California’s deputy CIO Adrian Farley contends it drives down costs by reducing vendors’ risk and allowing the state to determine whether their bids are realistic before signing. “It’s our firm belief that you get lower cost and quicker delivery by having a vendor have better information prior to placing a bid,” he said. Case in point: California sank $1.5 million up-front into the state’s second attempt to rebuild its payroll system by awarding $500,000 each to three finalists to create prototypes. After a nine-week bake-off period, however, the winning bidder, SAP Public Services, came in at $79 million, more than $40 million less than an earlier estimate for the project of $120 million.

Such an up-front investment may not be possible for smaller states, but California has seen the program reap benefits that go beyond short-term savings to reduced risk in the long run.

As a bonus, the two-step process gives vendors an incentive to communicate with the state during the bidding process — which helps identify potential problems and forge a strong relationship before the contract is signed. “Coming from a history where vendors don’t want to share their proprietary software, there was a lot of give and take with the state really learning from vendors,” technology analyst Li said. “That doesn’t happen a lot in procurement.”

Hope in Georgia

If there is a bright spot in the future of outsourcing large-scale state computer networks, it may lie in Georgia where a major overhaul of the state’s IT infrastructure is under way. 

“All eyes are now on Georgia,” NASCIO’s Robinson said, “They are the next ones on the block.” So far, the $1.2 billion, eight-year contract with IBM and AT&T has hit all of its deadlines with little difficulty. The vendors have successfully taken over the state’s IT infrastructure and are now in the process of consolidating all agencies into a single statewide system.

A new state management agency, the Georgia Technology Authority, was created to manage the mammoth project and the contract linked vendor payments to achievement of project milestones. 

“We had our service management organization staffed and ready to go before our service providers hit the ground,” director Patrick Moore said. To make that happen, Moore said, “We spent a lot of time communicating with agency heads. There was nobody who could say we didn’t look them in the eye and tell them what we were doing and why we were doing it.”

Still, Moore said the biggest challenge has been holding vendors back from launching new phases of the project before agency managers were ready. He admits that there have been some “dips in service” since IBM took control of some 80 percent of the network in June 2009. But customer satisfaction surveys have been improving ever since, and the project is on-budget and only two months behind schedule.

Now comes the hard part — consolidating the networks of every state agency into a seamless whole. Moore said he hopes Georgia’s careful planning and early agency involvement will allow it to avoid the pitfalls that have bogged down so many other state projects.

No matter how well a vendor performs, it is up to the state officials managing the project to ensure that the new systems are capable of meeting the state’s obligations. “There is a misconception that when you outsource, the provider is going to come in and do all of the work,” Moore said. “It doesn’t quite work like that. The state bears a very large burden to make the deal work. At the end of the day, they are still the ones responsible for delivering the services.”