View full PDF report
Report Number: 2016-125.2
The University of California Office of the President The University of California Office of the President Increasing Costs and Scheduling Delays Have Hampered the UCPath Project and Originally Anticipated Savings Are Unlikely to Materialize

Audit Results

UCPath’s Cost Has Soared, and Its Expected Savings Are Unlikely to Materialize

Escalating cost estimates and eliminating a plan to reduce staff have negated all the estimated savings the Office of the President originally expected from implementing UCPath, resulting in the original expected net savings of $447 million becoming an estimated cost of $942 million. In its 2011 business case outlining the cost and benefits of implementing the new system, the Office of the President stated that the cost to implement UCPath would be significantly less than the savings from it. As Figure 5 shows, the Office of the President estimated UCPath’s implementation cost at $170 million, including software licenses, hardware and infrastructure, project staffing, vendor services, and start‑up cost for the UCPath Center. It also forecasted $48 million for software hosting and maintenance and for technology cost, and $15 million for facility and equipment cost for the shared service center. In addition, because of the expected reductions in payroll, human resources, and IT staff, the Office of the President included a cost of $73 million for staff severance and job transition. Together, these costs totaled $306 million, a significant investment of funds, particularly at a time when the university was experiencing financial constraints. However, the Office of the President assumed staff reductions would create significant savings to more than offset these costs.

Figure 5
The UCPath Project Will Not Realize Cost Savings From Staffing Reductions as the Office of the President’s
2011 Business Case Projected
(In Millions)

Nonetheless, as the UCPath project encountered delays and other difficulties, its estimated implementation cost increased significantly, more than doubling by April 2017. At that time, the Office of the President reported that the estimated implementation cost for UCPath had risen from $170 million to $504 million, an increase of $334 million. Additionally, the Office of the President purchased the UCPath Center building and improved it at a cost of $53 million while its estimates show the cost to operate the UCPath Center through Wave 4 deployment at another $130 million. Although the university will no longer incur the staff severance cost of $73 million because the Office of the President no longer foresees a reduction in university staff, that savings is offset by the Office of the President’s greatly underestimating the cost that each of the 10 campuses will incur to implement UCPath. In its 2011 business case, the Office of the President estimated campus costs at $56 million of the implementation cost of $170 million; however, the 10 campuses reported to us that they expect to incur total costs of $193 million, minus $53 million in reimbursements from the Office of the President, as Table 3 shows. The Office of the President recently surveyed the campuses and other university locations and arrived at implementation costs of about $217 million, excluding the $53 million in reimbursements. Even using the more conservative cost total the campuses reported to us, the total estimated cost of UCPath stands at $942 million as Figure 5 shows. Should the UCPath project encounter further delays—which our IT project management expert believes is a risk given the project management weaknesses we discuss later in this report—the project’s cost will most likely increase further.

Table 3
Hallmark’s Program Manager Does Not Appear to Possess the Qualifications That DWR Required When It Selected URS

							FORECAST
	FY 2011–12	FY 2012–13	FY 2013–14	FY 2014–15	FY 2015–16	FY 2016–17*	FY 2017–18	FY 2018–19	TOTAL
Wave 2 (December 2017)
Los Angeles	$1,227	$4,363	$5,250	$4,961	$9,203	$9,726	$8,444	NA	$43,174
Merced	408	640	2,239	1,784	1,339	1,843	1,227	NA	9,480
Riverside†	134	651	834	1,016	2,447	6,967	8,713	NA	20,762
Wave 3 (July 2018)
Davis	$209	$1,139	$3,766	$3,405	$2,910	$3,923	$6,447	$2,995	$24,794
Irvine	423	616	1,327	1,769	1,849	3,186	5,117	2,408	16,695
Santa Barbara‡	111	406	834	1,294	1,474	2,359	5,140	3,218	14,836
Santa Cruz	242	1,341	2,604	3,915	1,823	1,078	5,850	3,150	20,003
Wave 4 (December 2018)
Berkeley	$260	$728	$833	$180	$0	$86	$8,742	$9,178	$20,007
San Diego	764	1,331	1,331	858	931	1,022	3,395	3,941	13,573
San Francisco	87	176	603	860	1,030	497	4,919	1,477	9,649
Totals	$3,865	$11,391	$19,621	$20,042	$23,006	$30,687	$57,994	$26,367	$192,973
Less reimbursements $53,000
Net total $139,973

In its 2011 business case, the Office of the President asserted that UCPath, as one of the projects under the Working Smarter initiative, would achieve sustainable long‑term cost savings including the $753 million saved from staff reductions. Additionally, the 2011 business case indicated that UCPath would provide significant benefits that were not monetized, including making the university’s payroll process more efficient, improving its payroll stability by replacing the legacy payroll system, and unifying its payroll and human resources into one IT system. The Office of the President viewed UCPath as part of a larger vision to improve the university’s administrative and operational effectiveness and as a platform for its campuses to adopt new and effective business processes.

Although the Office of the President still expects to achieve the nonquantifiable benefits, it no longer expects to realize the projected savings of $753 million from UCPath’s implementation. The project director indicated that it was not clear whether campuses would make any staff reductions or if such reductions—if they did occur—would be quantifiable. The three campuses we visited confirmed that they would not make staff reductions when implementing UCPath. For example, the former UCPath pilot director at the Riverside campus said that his campus has no plans for staffing reductions; rather, the campus plans to use any staff reduction from UCPath to offset the need for additional staff as the campus grows. Similarly, the Berkeley and Irvine campuses both indicated that they had no immediate plans for staff reductions and told us that they would better understand the impact of UCPath on their current staffing levels as they neared deployment.

Without the staffing reductions the Office of the President’s 2011 business case outlined, UCPath is not likely to result in significant savings in the near future, if at all. The failure of this $753 million in savings to materialize is a major change in the economic impact of the project on the university, since the Office of the President projected UCPath would provide a net quantifiable benefit of $447 million after its implementation cost as shown in Figure 5. Now UCPath will cost at least $942 million, placing further strain on the university’s financial condition. In July 2017, the Office of the President presented to the regents a UCPath project status report that included a section titled “Validating the Business Case for UCPath.” At present, the Office of the President is not generally focused on UCPath resulting in savings but rather on it creating efficiencies and avoiding unnecessary costs. In the status report, the Office of the President indicated that the campuses could identify efficiencies from centralizing and automating business practices. However, as noted previously, the three campuses we visited do not expect staff reductions that would drive the level of savings the 2011 business case envisioned.

The Office of the President Did Not Keep the Regents Apprised of the Significant Cost Increases and Schedule Delays That Have Plagued UCPath

Although the UCPath project has significantly exceeded its original cost estimate and schedule goals, the Office of the President has not provided timely or consistent updates of these changes to the regents. With cost increases of $334 million—from $170 million to $504 million—and four time extensions on completion—totaling nearly five years, from August 2014 to June 2019—we expected the Office of the President to have regularly updated the regents and any other stakeholders. However, in most instances, the Office of the President did not communicate these changes to the regents either before or at the time it made them. As a result, the Office of the President has effectively limited the regents’ governance role and impeded their ability to critically evaluate UCPath and whether committing hundreds of millions of dollars to an IT system was in line with the university’s financial goals.

The regents meet at least six times a year, in two‑day meetings every other month. These meetings have provided the Office of the President ample opportunity to communicate with the regents about UCPath. Although the Office of the President provided us a list of 79 communications to the regents that reference UCPath, we were able to find only five status updates on UCPath among these 79 communications. In the remaining 74 communications, the Office of the President either mentioned UCPath in relation to other issues or did not report any substantive details about the project’s status, cost, and schedule. In addition, the Office of the President updated the regents twice more after it provided us the list of 79 communications. The Office of the President provided a written update in April 2017, following our inquiries into why it had not updated the regents on UCPath’s cost increases and schedule delays. The Office of the President also updated the regents at the July 2017 regents meeting. All seven of these updates are reflected in Figure 6, along with a financing request the Office of the President made in May 2013, which mentioned UCPath’s then‑current cost estimate. The infrequency with which the Office of the President provided the regents with updates is troubling given the complexity of the UCPath project and its escalating cost.

Figure 6
The Office of the President Did Not Consistently Inform the Regents of Changes to UCPath’s Implementation Cost and Schedule

Further, the Office of the President did not disclose in any of the seven updates or in the financing request that it no longer believed the cost savings of $753 million would materialize. The absence of discussion on failed cost savings in the updates is a serious omission given that the Office of the President directly aligned UCPath with its Working Smarter initiative, through which it sought administrative efficiencies systemwide by reducing costs or increasing revenues.

Moreover, our review of the Office of the President’s limited communications with the regents found that the Office of the President did not always clearly identify UCPath’s current cost estimate or the timeline for completion. The Office of the President’s March 2013 status update to the regents referred to the UCPath project as “7 or 8 percent over budget” when in actuality it was 30 percent over its cost estimate, or about $51 million, at that time. Two months later, in May 2013, the Office of the President sought the regents’ approval for financing to fund the project’s escalating cost and revealed that UCPath’s cost estimate had increased to $221 million. The Office of the President discussed the status of UCPath three more times with the regents between July 2014 and January 2016 but was not forthcoming in two of these meetings. In July 2014, the UCPath project director told the regents that UCPath’s cost estimate was $220 million, yet the Office of the President’s internal records from one month earlier show the project’s cost estimate was $345 million. The UCPath project director also told the regents that the Office of the President would propose a final budget in the regents’ January 2015 meeting, but it failed to do so. The Office of the President did not provide the regents with the next status update until July 2015, when it reported the estimated cost of the project had increased to $375 million, which was in line with the Office of the President’s then‑current internal cost estimates.

In July 2015, the regents expressed concern about the rising cost of UCPath and directed the Office of the President to keep it informed about the project’s progress. Despite this specific direction from the regents, the Office of the President did not make its next status report until six months later, in January 2016. Further, in this status report, the Office of the President made no mention of the project’s cost estimate, which its internal records indicate had increased by another $60 million, or 16 percent, to $435 million. In fact, the Office of the President failed to inform the regents of UCPath’s escalating cost estimate and schedule delays until after we asked in February 2017 why it had not communicated this critical information. In April 2017, it finally sent a status update to the regents, its first substantive communication on UCPath’s status since the regents’ directive in July 2015.

The Office of the President’s April 2017 update revealed its lack of transparency on reporting UCPath’s true cost and schedule. In this update, the Office of the President reported that the estimated cost for UCPath had risen to $504 million, which it generally attributed to the need for increased time for campus readiness activities, additional staff for deployment activities, modest software customizations, and a four‑month extension for the pilot deployment. In a footnote to that update, the Office of the President noted having not reported UCPath’s cost estimate to the regents for nearly two years, with its last update occurring in July 2015 when it reported that UCPath would cost $375 million. The footnote offered no explanation as to why the Office of the President failed to keep the regents informed about the escalating cost of UCPath. Further, although the April 2017 update discussed the timeline for deploying UCPath at the campuses, it did not disclose that it had already pushed back UCPath’s final completion date to June 2019, which is when post-deployment support ends and UCPath is fully deployed.

Figure 6 shows that the Office of the President did not regularly communicate its decisions to change UCPath’s implementation schedule. On four occasions, the Office of the President extended the UCPath project schedule by six to 28 months, but it did not apprise the regents of three of these extensions or the reasons for the delays. In total, the Office of the President extended the length of the UCPath project from three years to over seven years. Although its status updates did provide estimated deployment dates for different campuses and other university entities and included some general reasons for extending the timeline, the updates were frequently vague about the project’s final completion date, again limiting the regents’ opportunity to critically evaluate UCPath’s direction.

We believe the Office of the President’s lack of transparency with the regents can largely be attributed to a weakness in the project’s governance structure, which has not included a process to keep the regents informed about UCPath’s progress. The project’s governance structure invests a great deal of autonomy and authority in the project’s executive leadership team (leadership team), which consists of the project sponsors, representatives from the Office of the President, and representatives of the campuses. Figure 3 shows the project governance structure. The bylaws of the leadership team state that it is responsible for final decisions regarding the UCPath project’s cost, schedule, scope, and policy. However, the bylaws contain no mention of communication with the regents: for example, they do not establish how frequently the leadership team must update the regents or what events should trigger updates about the project’s progress. In addition, the communication plans that the project’s governance and communications lead provided us lacked mention of periodic, structured communication with the regents. We find the absence of such a reporting relationship surprising, given the significant impact of UCPath on the university. The project director and the project sponsors confirmed that until recently the Office of the President did not have guidelines for communicating with the regents about the status of UCPath. While the project director acknowledged that the project sponsors have primary responsibility for updating the regents at significant implementation points, our findings show that they have not adequately fulfilled this responsibility.

In July 2017, the Office of the President informed the regents of a change in the UCPath project governance approach that it believes will address its lack of communication. Specifically, the Office of the President updated the UCPath Governance Materiality and Escalation document (governance document) to include the regents in the governance. The approach recognizes the regents as providing institutional oversight and identifies three situations that will require the project sponsors to update the regents about UCPath’s status. At the July 2017 meeting with the regents, the project director acknowledged that project governance decisions about UCPath had previously stopped with the project sponsors. He stated that in modifying the project’s governance document and acknowledging the regents’ oversight role, the Office of the President was responding to changes that the regents had made in 2016 to the charter for the Finance and Capital Strategies committee—a committee of the regents with oversight of the university’s fiscal and financial affairs and business operations, among other matters—to reflect that committee’s oversight of UCPath as a large‑scale enterprise system. We note that the timing of the governance change coincided with this audit and the inquiries we made about the Office of the President’s infrequent communications with the regents about the status of UCPath.

Although the Office of the President has now acknowledged the regents as part of the governance for UCPath, this expanded governance approach may not go far enough to ensure that the regents can participate in critical decisions. The governance document states that updates to the regents must occur when the project meets significant implementation milestones, including deployments; when the project experiences adverse cost impacts of more than $20 million; or when the schedule is delayed by more than three months. However, the governance document makes clear that the regents will be informed or “kept apprised of progress or changes via one‑way communication” rather than consulted. In our view, this governance approach does not adequately recognize the regents’ role as an oversight body. We believe the Office of the President should address the need for the regents to have timely, critical information in order for them to participate in decision making and to provide the project with guidance. Although the Office of the President has committed to informing the regents of certain project events, which may cause it to report more frequently to the regents than in the past, the planned approach will not engage the regents on a decision‑making level that would truly allow them to fulfill their oversight role.

Weaknesses in the Office of the President’s Project Management Likely Contributed to UCPath’s Cost Increases and Schedule Delays

Our review found that the Office of the President could have mitigated cost and schedule increases to the UCPath project through better project management practices. Specifically, our IT project management expert identified the following five weaknesses in the Office of the President’s project management processes:

• Maintaining an overly aggressive project schedule.
• Not maintaining an integrated schedule and resource management plan (integrated plan).
• Not assessing all budget and schedule implications resulting from change management.
• Not establishing effective risk‑management processes.
• Not using independent verification and validation (IV&V).

The Office of the President maintained an overly aggressive project schedule, which became susceptible to delays because of project scope changes and staffing constraints. When our IT project management expert reviewed the UCPath project’s risk log, the project sponsors’ briefings to the president, and the project director’s briefings to the project sponsors, he identified a pattern: the UCPath project schedule became too aggressive and then slipped. In fact, the project briefing materials demonstrate that the Office of the President was aware that the schedule was aggressive: mentions of it appear time and again in those materials. For example, an August 2014 briefing prepared for the president stated, “Allotted time to prepare for [the Office of the President] go‑live is aggressive. ”According to our expert, go‑live refers to the point in time that a new IT system is put into use, or production; thus, this briefing reveals that the Office of the President was aware that it might not be able to deploy UCPath as planned. Similar concerns show up in many briefings, yet the Office of the President continued to set aggressive schedule goals.

Additionally, when the project director provided monthly briefings to the project sponsors, he often indicated that there was no schedule slack—a term that refers to the amount of time that critical tasks can be delayed before the schedule is jeopardized. For example, in August 2016, the project director reported that the numerous changes that the campuses had requested could cause schedule delays. And a January 2017 briefing indicated that the project had no slack in the schedule and that significant overlapping work might cause problems. Overlapping work in the UCPath schedule is a concern because it indicates that the Office of the President has scheduled the same core staff to perform two different tasks at the same time. These issues led our IT project management expert to conclude that the project’s overly aggressive schedule factored into the project’s repeated delays.

Further, our IT project management expert found that the Office of the President did not maintain an integrated plan. According to our expert, building and maintaining an integrated plan is critical for a project of UCPath’s size and complexity. Such a plan captures the timing of each project task and identifies which staff will perform that task and helps identify and resolve instances where staff have overlapping work before they impact the project’s schedule. When our expert reviewed the project briefing materials, he found that the Office of the President had acknowledged the risks associated with overlapping schedules and had identified the lack of an integrated plan as a risk to the UCPath project schedule. According to the current project director, between 2013 and 2014 his former codirector attempted to develop and maintain an integrated plan, but the planning software that the Office of the President was using, as implemented, could not support the level of detail the plan required. Subsequently, the plan was abandoned. Our expert found no evidence that the Office of the President attempted to adopt a subsequent integrated plan. Further, he observed that many other large IT projects have used integrated resource and schedule planning without the technology problems that the Office of the President encountered.

Our IT project management expert also observed that the Office of the President lacked a rigorous process for assessing the budget and schedule implications of proposed scope changes to the UCPath project. Although the project has a defined process for large changes, those changes that are believed to be smaller are approved at the lowest project levels without assessment of higher or global implications. Over time, small changes accumulate and have more significant project impacts. Without a detailed analysis of how changes may affect the broader project context, staff or management may approve small changes without understanding that those changes will have ramifications that threaten the project’s larger goals. Changes to UCPath’s scope were likely to have implications for its cost and schedule because it has operated with aggressive schedules, without the necessary staff to accomplish all planned work, and with a work plan that involves overlapping tasks.

The Office of the President also demonstrated weaknesses in its risk‑management processes. The purpose of risk management is to decrease the likelihood and impact of negative events threatening project goals. Risk management is accomplished through processes that identify, prioritize, and develop responses to threats to those goals. Although the Office of the President tracked risks, it did not review or monitor all risks periodically. Specifically, our IT project management expert found that the project risk logs identified many medium‑severity risks to the project’s schedule, yet many of those risks sat for hundreds of days without evidence of reassessment—often at the same time that the project schedule was slipping. For example, the risk manager logged in September 2016 that a specific testing environment had not been fully evaluated before use and could result in defects during testing. However, this risk went unmonitored for more than four months with no apparent actions or updates. Without regular monitoring of identified risks, the Office of the President may not detect which risks have passed, which have been mitigated or resolved, or which may become more significant threats.

Finally, the Office of the President also failed to use IV&V as part of its project oversight. IV&V helps to ensure that an IT system will perform as intended and meet its users’ needs. As an IT system is developed and implemented, IV&V can provide early warnings of process and technical discrepancies, issues, and problems that may otherwise go undetected or be detected too late. According to our IT project management expert, UCPath’s size, complexity, and risk warranted using IV&V for identifying and mitigating technical and project management issues. For example, IV&V could have detected the issues UCPath experienced with staff being over allocated. One of the project sponsors asserted that the Office of the President’s internal auditors and a consultant have been providing oversight that is equivalent to IV&V. Specifically, she confirmed that the internal auditor works with the consultant to create a report of any significant risks to the UCPath project. However, our expert concluded that neither of these entities provided oversight that was equivalent to IV&V, as IV&V provides a more robust analysis of technical activities and is embedded in the project as opposed to occurring periodically.

In addition to the project management issues that our IT project management expert identified, the Office of the President provided several factors that it believes contributed to its exceeding its initial UCPath cost estimate and schedule goals. Specifically, the UCPath project director acknowledged that the Office of the President greatly underestimated UCPath’s size and complexity, particularly in terms of the changes required to adapt the system to each campus’s business processes. Our expert concurs and indicates that the significant challenges the project has encountered were indicative of the Office of the President’s poor initial understanding of the UCPath project’s scope. These challenges have not only included missed schedules and a growing budget, but also an increase in the predicted number of necessary system interfaces between UCPath and other IT systems from 75 to over 120. Each interface requires analysis and software programming, which added to the project’s complexity.

The Office of the President also did not fully understand the amount of effort required to standardize business processes across all campuses. One of the project sponsors, the chief financial officer, told us that a main lesson the Office of the President has learned is that UCPath is a business transformation project—meaning the university has to change its business processes systemwide—as well as an IT implementation project. He indicated that the Office of the President made the mistake of procuring a technological solution for the university’s payroll and human resources activities before it had standardized its business processes and developed a shared service center model for those activities.

The project director also explained that schedule delays and cost estimate increases are closely related because labor costs account for the majority of UCPath’s implementation cost; thus, a delay in the overall schedule results in an increase in UCPath’s cost estimate. Further, he indicated that the Office of the President is constrained by the university’s business calendar, which offers four optimal dates to shift from one IT system to another. These dates coincide with the end of each tax reporting quarter and the project director stated that deploying UCPath at the end of a tax quarter minimizes the need to convert employees’ tax balances. Thus, when the schedule slips, it slips in three‑month increments, which means the university incurs labor costs for three more months, resulting in large increases in UCPath’s cost estimate. Ultimately, although the Office of the President is aware of some of the factors that led to the project’s cost increases and schedule delays, such awareness has not prompted addressing the project management weaknesses that our IT management expert identified. The Office of the President needs to develop and implement IT project management guidelines that will help ensure that IT projects are completed on time and within approved budgets.

Although Invoice Approval Processes Were Followed, Some Campuses Could Better Define IT Deliverables

To assess the university’s management of its IT contracts, we reviewed one IT project at each of the Berkeley, Irvine, and Riverside campuses. We found that while the campuses followed their established vendor payment practices, the Berkeley and Irvine campuses could improve their IT contracting practices. The projects we reviewed included fixed price contracts with development vendors (Berkeley and Irvine campuses) and a time‑and‑materials contract with a development vendor (Riverside campus).  The costs of the three projects ranged from $11 million to $93 million. Our review of five project invoices from each campus found that the campuses had required that vendor invoices be reviewed and approved before payment. However, the Berkeley and Irvine campuses had only vaguely worded deliverables for project milestones. According to our IT project management expert, these types of vaguely worded deliverables do not align with industry best practices and increase the risk that the campuses will inadvertently pay for work that does not meet their expectations or needs.

The State has established guidance on IT contracting that helps ensure that IT contracts contain well‑defined deliverables. For example, the State Contracting Manual states that agencies must develop a “clear, concise, and detailed description of the IT services to be performed.” Further, the State’s Project Management Framework (management framework) provides guidance for how to define deliverables to ensure that the agency and the vendor possess a mutual understanding of the content and scope of the deliverables. This sort of clear definition helps to ensure that the agency gets what it is paying for: an IT system that functions as intended. The management framework recommends that the agency develop a deliverable expectations document that defines the scope, content, entrance criteria, acceptance criteria, and development schedule for each deliverable. The entrance criteria should define what the vendor must achieve before it begins work on activities associated with the development of the deliverable, and the acceptance criteria should define what the vendor must achieve before the agency will accept the deliverable. Finally, the deliverable schedule should document the key tasks and dates associated with the deliverable.

Our review found that the deliverables for the Berkeley and Irvine campus projects did not always align with the management framework’s guidance. Although state law does not require the university to comply with these guidelines, they represent best practices that it would likely benefit from. We found that the Berkeley and Irvine campuses paid their vendors on the completion of milestones; however, they did not have deliverable expectation documents, and the deliverables for milestones as described in the vendor contracts did not provide enough detail to effectively measure whether the vendors had met their obligations for payment. For example, although the contracts had defined dates for the completion of milestones, they did not include entrance or acceptance criteria. Therefore, the Berkeley and Irvine projects did not have sufficient criteria to determine if the milestones had actually been completed.

Absent sufficient criteria, it is difficult to determine whether the vendor achieved a deliverable and whether that deliverable was acceptable. For example, in January 2016, the Berkeley campus’s vendor submitted an invoice of $593,000 for a milestone titled Enrollment History Converted, and the Berkeley campus subsequently paid that vendor for completing the milestone. However, the Berkeley campus’s basis for making this payment is unclear because the contract does not define the deliverables that would satisfy the milestone. Further, the contract leaves in question what level of data conversion is acceptable, such as whether the campus will accept a certain percentage of errors in the data or a certain percentage of data that are not successfully converted.

The executive director of the Berkeley campus project responded that the Berkley campus has a comprehensive process for accepting milestones and is aware of what each milestone includes based on what he stated is a highly detailed project plan and a readiness assessment. Similarly, the director for the Irvine campus’s project stated that the Irvine campus uses the project schedule and the statement of work to determine whether to pay its vendor for achieving milestones. Further, he told us that the Irvine campus had not made payments for some of its vendor’s invoices because the campus did not believe that the vendor had satisfactorily completed certain milestones. Despite the processes that the Berkeley and Irvine campuses assert that they follow, our IT project management expert found that their contracts did not have defined milestones, which puts them at risk for contractual disputes with their vendors.

The Berkeley campus’s opting not to employ best practice in its process for deliverable review did not appear to affect the cost of its project. In 2017 the Berkeley campus completed most of the project development activities within its budget of $93 million. In contrast, the Irvine campus is in the early stages of developing its Student Information System and has struggled to keep the project on track, which is projecting a $12 million overage from its initial budget and a one‑year delay. Thus, it could benefit from improving its deliverable review practices.

Recommendations

Regents

To ensure that they are able to exercise necessary oversight for the university’s significant IT projects, the regents should develop status reporting standards for the Office of the President and all university locations to follow by December 2017. Such reporting standards should apply to all university IT projects with more than a specified cost and, at a minimum, should establish the following:

• The frequency with which the Office of the President and all university locations must report to the regents. Such updates should occur at least three times per calendar year and coincide with regents’ meetings to allow for oral discussion.
• The types of disclosures the Office of the President and all university locations must present about each IT project including, but not limited to, changes in scope, projected cost, and schedule.
• The types of significant project risks the Office of the President and all university locations must disclose. The updates should also describe the actions the Office of the President and all university locations are taking to mitigate risks and the potential effects of those risks on a project’s cost, schedule, and scope.

Office of the President

To ensure that it fully reports the cost of IT projects, the Office of the President should develop cost reporting guidelines by December 2017 for UCPath and other significant IT projects across all university locations. These cost guidelines should identify cost categories at both the Office of the President and university locations to ensure that the estimates capture and communicate all development and implementation costs. In addition, the Office of the President should produce cost reports to share with stakeholders at least quarterly.

To ensure that it consistently follows best practices related to project management, the Office of the President should develop and implement guidelines for IT project development by June 2018. The guidelines should apply to all IT projects undertaken by any university location with a cost estimate of at least $5 million or more and should include the following elements:

• A means to assess schedules for reasonableness, and requirements for the creation and maintenance of an integrated schedule and resource plan for each project.
• Requirements for rigorous change management processes that establish a means of assessing the implications of changes to a project’s scope, cost, and schedule.
• Procedurally sound requirements for identifying, reviewing, and resolving risks to a project.
• IV&V to oversee the technical aspects of project development.

The Office of the President should require that all university locations follow best practices by ensuring that each location creates a deliverable expectations document for each IT contract similar to the documents the State’s management framework describes. The Office of the President should establish this requirement by December 2017. The deliverable expectations document should, at a minimum, identify the deliverables for each milestone and define the scope, content, entrance criteria, acceptance criteria, and development schedule for each deliverable.

We conducted this audit under the authority vested in the California State Auditor by section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the Scope and Methodology section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Respectfully submitted,

ELAINE M. HOWLE, CPA
State Auditor

Date:
August 24, 2017

IT Audit Support:
John Baier, CPA, Audit Principal
Sharon L. Fuller, CPA
Idris H. Ahmed

IT Project Management Expert:
Catalysis Group

Legal Counsel:
Joseph L. Porche, Staff Counsel

For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255.

---

Back to top