Reforms needed for state technology to work

Posted December 19, 2013

State Chief Information Officer Chris Estes demonstrates prototypes of new government agency websites in the Innovation Center on Jones Street Dec. 13, 2013.

— In early 2008, two men kidnapped and murdered Eve Carson, the student body president at the University of North Carolina at Chapel Hill, only a mile away from campus.

Her death shocked the university community and exposed a glaring gap in the state's ability to keep track of those on probation and parole.

Carson's killers – both of whom had been charged with serious crimes months before – slipped through the judicial system because information that would have raised red flags was buried in piles of paper or simply inaccessible in the courtroom. The case spurred state lawmakers, bureaucrats and law enforcement officials to create CJLEADS, a computer system that matches and organizes information collected by a number of agencies, from police to wildlife officials, and dispenses it in quick, easy-to-understand bites.

"We had a very compelling event that happened that caused the business partners to have a vested interest in a solution," said Kay Meyer, the program director in the Office of State Controller who oversees CJLEADS, which stands for Criminal Justice Law Enforcement Automated Data Services.

 Troubled Tech

Thursday's story is the second of a two-part series on the systemic problems with state technology projects. See part one for an in-depth look at the problems that have plagued large IT initiatives in North Carolina for years.

Having a clear purpose that everyone from lawmakers to users could understand helped keep CJLEADS on track from its inception, Meyer said. It's just one of the core principles technology experts say could help other state-run information technology projects often plagued by inaccurate estimates, lopsided contracts and inadequate oversight.

After a year in which audits, public hearings and investigations pilloried a number of these projects, state leaders and technology consultants say it's time to learn from both successes and failures to avoid high profile and expensive missteps going forward.

To succeed, avoid the 'big bang' approach

The first version of CJLEADS didn't do everything Bob Brinson would like it to. It still doesn't.

And that's a good thing.

Brinson, the chief information officer at the state Department of Public Safety, knows the system already helps probation and parole officers do their jobs more efficiently by giving them offender data that cuts across a number of different bureaucracies.

Chris Estes, state CIO Custom tech can be a pitfall

He would like to tie CJLEADS' capabilities into a program built specifically for parole officers, allowing one computer system to display data housed by the other. But that kind of integration would require thousands of hours of work to program and test and was not part of the original specifications for CJLEADS.

That's why neither the controller's office nor the cross-department committee guiding the system's development have acted on his request – at least not yet.

"They have been pretty ruthless about saying, 'We get that's a future need, but for right now, it's not part of the core system,'" Brinson said.

The agency's disciplined focus is often missing from other more problematic systems derailed by demands for new features that keep them from being built on time and on budget.

Rather than having to turn one mammoth system on all at once, state Chief Information Officer Chris Estes said any IT project will be more likely to succeed if it can be built and launched in pieces. 

"Doing this in incremental, small steps versus kind of a 'big bang' is what the private sector does and what I'm trying to get done here," Estes said.

Taking smaller steps would also avoid projects sprawling from one gubernatorial administration to the next, forcing different groups of leaders to try to ride herd on projects started by predecessors. 

Meyer said the state has rolled out two or three updates to CJLEADS every year, adding on new data sources or functions in small bites.

For example, as it was first rolled out, officers could access the system only through laptop or desktop computers. Today, a web application allows parole officers and police investigators to access CJLEADS at their desks, in the car or walking up to someone's door.

It was a feature users not only asked for but helped design, Meyer said.

That culture of user involvement in the design and development of a system – nailing down what the technology needs to do – is what Estes said he's trying to make a standard part of government IT projects.

Getting requirements right

When Gov. Pat McCrory first took office in January, he decried the state of the North Carolina's computer infrastructure. Outdated desktop computers, a fire in a computer equipment closet and sprawling projects behind schedule greeted the new chief executive in 2013.

Chris Estes, state CIO

During his campaign and just before he was taking office, Gov. Pat McCrory placed a high priority on getting the state’s information technology systems up to modern standards and getting technology problems under control.

“It is clear we have to modernize and digitize state government,” McCrory said at the time.

So it was no surprise that one of his first high-profile hires, introduced at a January news conference, was Chris Estes, who now holds the title of North Carolina’s chief information officer.

“He’s going to be spending a lot of time with the cabinet secretaries,” McCrory said at the time. And indeed, although he doesn’t hold cabinet-level rank, Estes sits at the governor’s weekly cabinet meetings and is one of the most lauded individuals in the governor’s leadership team.

During the first year of McCrory’s term, problems with large IT systems that handle Medicaid and food assistance payments have dominated headlines. Meanwhile, state government is grappling with technology that has not aged gracefully, such as desktop computers with operating systems that their manufacturers no longer support.

“With Chris Estes in place, we will get our hands around this, resolve and fix this,” said Sen. Jeff Tarte, a Mecklenburg Republican with experience in the information technology field. Tarte said Este brings the “specific, detailed, experience in IT management in large complex systems” that the state needs.

As CIO, Estes has statewide responsibilities for buying, maintaining and coordinating information technology. Before coming to the state, Estes was a principal at Booz Allen Hamilton. He has experience in IT working for manufacturing, IT and financial services companies such as IBM, BearingPoint and Price Waterhouse.

Estes said the economy, particularly the recent recession that shaved government revenues, was to blame more than any group or individual.

In 2008, he said, the state set aside $20 million a year for information technology upkeep and oversight. By the time McCrory took office, that fund was down to $6 million. Much like aging state buildings, state trooper cars climbing over their recommended mileage and dozens of other infrastructure and equipment issues, a lack of funding had eroded North Carolina government's IT backbone.

"The great recession has impacted all of us in ways we never anticipated," Estes said as he stood in the middle of North Carolina's Innovation Center, a suite of conference rooms equipped to test technology, which has been set up on the first floor of the the Department of Environment and Natural Resource's year-old building on Jones Street in downtown Raleigh.

The Innovation Center is part of the administration's effort to avoid problems with big technological projects. A key function of the space, Estes said, is allowing end-users of new technology to test systems before they're deployed, and in most cases, before the state make a decision to buy. 

"If we're contracting through a competitive bid process and the vendor says their technology can do (something), part of that bidding process will include running it through here. So, we're going to see it work," Estes said.

Showing end-users the look and feel of potential products also allows them to make suggestions – and that can be helpful in the final design.

For example, it was a group of law enforcement officers who asked that data from the state Wildlife Resources Commission, particularly hunting permits, be added to CJLEADS. Meyer said offenders might try to evade a court official or lack the kind of utility bills that make them easy to find.

"If you happen to register for your hunting license, you generally do want to receive it, so you give a good address," Meyer said.

State Auditor Beth Wood, whose agency produced several audits ripping the state for a disjointed technology contracting process, said earlier and more meaningful user involvement would have helped avoid many of the costly issues her department identified by clearly defining each system's basic requirements before the bidding and contract process.

"Sometimes, we're trying to force a square peg in a round hole," Wood said. "We're just not, No. 1, stepping back and seeing what is it that we really need, who are the users and getting everybody to sit at the table. And it's not, 'What we want?' but 'What do we really need?'"

Getting better advice  

But buying a large IT system is not the same as contracting to renovate office space or buy a fleet of police cruisers, said Sen. Jeff Tarte, R-Mecklenburg, whose career has revolved around building and using information technology systems.

Auditors' pointers

In April 2013, the Office of the State Auditor released an audit of about 100 state technology projects recommended a list of changes to help improve the process of developing IT systems. CIO Chris Estes, of the Information Technology Services Division, agreed with nearly all of them, save ones requesting new laws. Here are the biggest ones.

  • ITS should develop and publish written guidance for developing state agency IT project costs and estimates.
  • ITS should require state agencies to obtain independent validation of the accuracy and reasonableness of IT project estimates.
  • ITS should request that lawmakers consider legislation to hold state agency managers accountable for cost and schedule estimates.
  • ITS should develop a method to identify state agency IT projects that require state CIO approval.
  • ITS should consider asking the General Assembly for the authority to ensure that ITS receives project status reports on schedule.

North Carolina, he said, has lacked expertise needed to ensure its technology contracts don't allow vendors to take advantage of taxpayers.

"This is just an open-ended, all-consultants-full employment act right now," Tarte said. "We leave ourselves totally open to being abused by vendors."

To tackle this kind of problem, experts inside and outside of government say North Carolina needs to hire temporary specialists to advise on the scope of work that's needed and help craft contracts with the ultimate vendor.

"Especially IT projects, where we don't have the talent in state government, you just go hire a subject matter expert," Wood said.

There needs to be a firewall between those experts and the ultimate bidders, she said. In recent cases, when the state has hired an expert consultant, that person has recommended themselves for the ultimate job.

"It needs to be perfectly clear that the person who was the subject-matter expert in helping the project get going and helping with the initial phase of it, that they are never allowed to bid on, nor participate in, nor anybody they're affiliated with be able to participate in," Wood said.

The idea is that the state is hiring an advocate, a person not unlike a lawyer or accountant, to help with negotiations.

Preparing for problems could lead to success 

Avoiding big change orders, getting better input from users, getting better advice from outside experts and anticipating problems aren't the only ways to get better results from state IT projects.

Dr. Albertina Smith at Sunshine Pediatric For state-funded technology, failure a likely option

Simply put, experts say, the state needs to develop better and more standardized systems for dealing with technology. Technology mavens in one agency don't necessarily use the same lexicon to talk about systems as the experts in other parts of government.

Wood's audits show similar findings. Her office's recommendation, endorsed by Estes, is for Information Technology Services to develop and publish standard guidance for estimating costs and schedules.

"There's basic management procedures and protocols for how to run large IT projects, and we've got to get back to some basics," Tarte said.

Both he and Estes, for example, said the state needs to pick and stick with one system for tracking the progress of a large IT implementation.

Wood said the state's in-house experts, Estes' ITS division, needs to be more involved in the oversight process. Too often in the past, she said, ITS staffers have merely signed off on reports from state agencies rather than providing true, independent oversight.

"Really, your fail-safe is ITS. They have the talent," Wood said. "They have the IT specialists over there. They are really the ones to make sure this project is on target, on track and on budget, because they know. But that's not been the case."

Estes agrees that his agency needs to take a more active role. Already, policies have changed to require chief information officers from various agencies to report in a more streamlined way to Estes and his agency.

Today, all new IT projects are assessed on a risk scale of one to three, with higher-risk projects getting more scrutiny – and more help – from the state's in-house experts.

But no one step guarantees success.

"They're never going to be perfect. If you do your research on commercial IT projects, they're usually over budget and not on time either," Estes said.

But the state, he said, can do a lot better to get up to the standards used by the business community.

"So, that would be our benchmark, is to at least get to industry standard," Estes said. "That would be a huge savings to citizens."

Thursday's story is the second of a two-part series on the systemic problems with state technology projects. See part one for an in-depth look at the problems that have plagued large IT initiatives in North Carolina for years.


This blog post is closed for comments.

Oldest First
View all
  • frogdog Dec 21, 2013

    I doubt Estes can find a way to make ITS effective. He was brought in all sorts of high paid consultants as direct reports and built that dog and pony show lab. Information is not flowing from him and his direct reports down to the employees. ITS employees are being demoted in title but not pay, converted to "serving at the will of the governor", and left not knowing what is going on most of the time.

    ITS still wants a one size fits all solution and wants agencies to use their services which cost more than private services. My agency has had to get the Gov's Office (under Easley) and the Budget Office (under McCory) involved in projects to get ITS to let us choose options that are the best price for the agency. It should not be that way. ITS should have the best interests of the state and the agencies which make up the state in mind.

  • tccostin2 Dec 20, 2013

    I've worked on both sides of the IT field within the State. I was a contractor for the state and a supplier of contractors, and everyone in the IT contracting field thought ITS was a huge waste of money.

    If ITS is going to be effective from this day forward, I feel they need to figure out how to best offer expertise without interfering with the agency. The idea of having one group to oversee all IT projects is incredibly short sighted. It's just not possible.

    How in the world can ITS know what is best for each agency? They can't. If Estes can find a way to make ITS an effective group that supports agencies with their IT projects, then I would applaud his efforts. Until then, ITS continues to be a HUGE WASTE of our tax payer dollars.

  • lexalexander Dec 20, 2013

    I'm not a computer engineer, just a user. But I use a lot and have for a long time. And on big projects, in my experience, incremental and iterative has always yielded better results than trying to make every moving part move at exactly the same time.

  • jonstewartrules Dec 20, 2013

    You could say the same thing about the State Legislature.

  • rasengineers Dec 20, 2013

    wlbbjb wrote, ""So, that would be our benchmark, is to at least get to industry standard," Estes said. "That would be a huge savings to citizens." And how much will this cost the taxpayer?"

    Yes it will cost money up front to implement project management systems comparable to some the systems some private entities are using; however, in the long run it will save money. The alternative is to continue experiencing huge cost over-runs, extensive time delays, and tolerating products that do not work.

  • teddyspaghetti Dec 20, 2013

    I have another suggestion.....how about make it mandatory that NC citizens are given preferential hiring over out-of-state or out-of country workers????

  • frogdog Dec 20, 2013

    While House Bill 700 has the potential to help with some of these problems, there is no clear definition from ITS about when you do not need to involve them. Do you really need to set up a full project, go through all of the reviews and gates, wait for weeks or months for a response from ITS when all that is really needed it s $2000 or less off the shelf product?

    Whenever the agency where I work has had to use ITS we have had project delays due to lack of ITS response, ITS turnover, lack of skills/knowledge within ITS, or ITS not caring about our projects. ITS likes to create one size fits all solutions. When ITS can finally realizes that not all agencies are alike or have the same amount of funding and can get skilled people and can have decent response times, then maybe the agencies would be more willing to work them.

    Until ITS has true changes that are for the benefit of the state and the benefit of politically connected individuals or companies then IT reform might begin.

  • wlbbjb Dec 19, 2013

    But the state, he said, can do a lot better to get up to the standards used by the business community.

    "So, that would be our benchmark, is to at least get to industry standard," Estes said. "That would be a huge savings to citizens."
    And how much will this cost the taxpayer?

  • timothyrpeck Dec 19, 2013

    State IT Governance Changes

    House Bill 700 requires that the State Information Technology Plan include a review of all state agency information technology projects — not just those costing in excess of $500,000, as the old law required.