Report 2014-116 Recommendation 28 Responses

Report 2014-116: California Department of Consumer Affairs' BreEZe System: Inadequate Planning and Oversight Led to Implementation at Far Fewer Regulatory Entities at a Significantly Higher Cost (Release Date: February 2015)

Recommendation #28 To: Technology, California Department of

To ensure that contracting departments comply with requirements, CalTech should periodically monitor the status of contracts and enforce adherence to procurement requirements. For example, CalTech should ensure that contracting departments forward all change requests to it and that it conducts reviews of all change requests to determine whether the changes exceed the scope of the respective contract.

Annual Follow-Up Agency Response From October 2018

SIMM 45 and the newly released Project Status Report (PSR), issued December 2017, specifically address scope changes and the management of change requests processes as a monthly review area of CDT's Oversight analysts. Furthermore, there is second level review as Oversight managers review all entities' PSR and IPORs.

The PSRs, which are initially reviewed by Oversight analysts, asks the following questions:

Variance detail chart:

1. Was work done that was not planned?

2. Were there any changes to the scope?

3. Were any deliverables or milestones removed?

Look Ahead view:

1. Is there any unplanned work that needs to be done?

2. Are there any expected or recommended changes to the scope?

3. Are there any deliverables that should be removed from the plan?

Additionally, the IPOR Assessment Criteria section in the IPOR template includes the following criteria for the Scope Management area:

- Are change requests are consistent with the approved scope of the project and are managed and monitored through completion and close out?

If any of these items identified above are perceived as a risk by Oversight analysts or managers, the Oversight framework (SIMM 45) specifically identifies our Statewide Technology Procurement Division as a point of escalation for Level 2. The Oversight manager is also directed to escalate unmitigated variances as outlined in the Project Oversight Escalation Process that was updated in December 2017.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

CDT has updated its assessment criteria within SIMM Section 45 regarding the contents of IPO reports to specifically state that IPO is to ensure change requests are consistent with the proposed scope of the project and they are managed and monitored through completion and close out. Further, its escalation process allows for the escalation of issues to its statewide technology procurement department and further specifies that change requests deemed to be outside of the approved scope or budget should be reviewed by its procurement department for validation. Thus, this recommendation is fully implemented.

Annual Follow-Up Agency Response From November 2017

CDT has updated the SPR preparation instructions to reflect potential alternatives if cost-benefit analysis is incomplete.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

In following up with CalTech regarding its implementation of this recommendation, we learned that it has not yet developed procedures to ensure that contracting departments forward all change requests to it and that it conducts reviews of all change requests to determine whether the changes exceed the scope of the respective contract. Until it develops such procedures, we will report this recommendation as pending.

Annual Follow-Up Agency Response From October 2016

ITPOD will examine change requests in the normal course of the oversight activity and if any are identified as exceeding the scope and intent of the current contract, ITPOD will raise that issue with the department and will bring the issue to the attention of the State Technology and Procurement Division (STPD), as needed. STPD will conduct a more detailed review to make the final determination whether or not the changes exceed the scope of the respective contract. Procedural documentation of these new oversight activities are expected to be completed by December 2016.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

It is important to recognize that change requests are often necessary and proper. ITPOD uses SIMM 45, Appendix F: Project Oversight Review Checklist to evaluate the conditions which necessitate changes to the contract as well as the processes for ensuring adequate decision making at the appropriate level for issuing a change request. This includes verifying that key specification documents (e.g. contracts, requirement specifications and/or contract deliverables) and software products are under formal configuration control, with items to be controlled and specific staff roles and responsibilities for configuration management identified in a configuration management plan. This also includes verification that a formal governance process is in place, which necessitates the documentation of the reasons for approval of change requests.

ITPOD will examine change requests in the normal course of the oversight activity and if any are identified as exceeding the scope and intent of the current contract, ITPOD will raise that issue with the department and will bring the issue to the attention of the State Technology and Procurement Division (STPD) as needed. STPD will conduct a more detailed review to make the final determination whether or not the changes exceed the scope of the respective contract.

California State Auditor's Assessment of 1-Year Status: Pending

In following up with CalTech regarding its implementation of this recommendation, we learned that it has not yet developed procedures to ensure that contracting departments forward all change requests to it and that it conducts reviews of all change requests to determine whether the changes exceed the scope of the respective contract. Until it develops such procedures, we will report this recommendation as pending.

6-Month Agency Response

The Department's Procurement division will continue to work with ITPOD to identify if a Change Request has deviated outside a procurement requirement. The division will then be notified to conduct a more detailed review to determine if the changes exceed the scope of the respective contract.

We expect to be able to report progress for the next scheduled update.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

To address the audit recommendation that CalTech review all contract requests, STPD has formally requested staff augmentation. We will be able to provide an update on the progress of this additional resource request in our 6-month status update report.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2014-116

Agency responses received are posted verbatim.