Report 2014-116 Recommendation 17 Responses

Report 2014-116: California Department of Consumer Affairs' BreEZe System: Inadequate Planning and Oversight Led to Implementation at Far Fewer Regulatory Entities at a Significantly Higher Cost (Release Date: February 2015)

Recommendation #17 To: Consumer Affairs, Department of

To the extent that Consumer Affairs chooses to implement BreEZe at the phase 3 regulatory entities, it should first complete a formal cost-benefit analysis to ensure that BreEZe is a cost-effective solution to meet these regulatory entities' business needs. To make certain this analysis is complete, it should include an assessment of the potential changes these regulatory entities may require to be made of the BreEZe system and the associated costs. Consumer Affairs should complete the cost-benefit analysis before investing any more resources into the implementation of BreEZe at the phase 3 regulatory entities, and it should update this analysis periodically as significant assumptions change.

Annual Follow-Up Agency Response From October 2021

The Department (department) of Consumer Affairs' Business Modernization Plan has successfully delivered initial technology releases for 6 of 16 boards and bureaus (boards) through 2021 and has achieved many planning milestones for the remaining boards. The boards that have deployed modernized online services as of September 2021 are the:

1. California Acupuncture Board,

2. Board of Chiropractic Examiners,

3. Board of Professional Engineers, Land Surveyors, and Geologists,

4. Court Reporters Board,

5. Bureau for Private Postsecondary Education, and

6. Contractors State License Board. The Contractors State License Board implemented their modernized online services after the last reporting period.

Since the last update, two additional boards have completed business process mapping: the Board of Accountancy and the Speech-Language Pathology & Audiology & Hearing Aid Dispensers Board. Also, one additional board has completed the Project Approval Lifecycle (PAL) Stage 1: the Speech-Language Pathology & Audiology & Hearing Aid Dispensers Board. Finally, six boards completed their PAL Stage 2 since the last update: the Speech-Language Pathology & Audiology & Hearing Aid Dispensers Board, the Cemetery and Funeral Bureau, the Bureau of Household Goods and Services, the California Architects Board/Landscape Architects Technical Committee, the Structural Pest Control Board, and the Bureau of Automotive Repair. The chart below reflects the current progress.

Table 1. Phase 3 PAL and Business Activities Status

Phase 3 boards/bureaus with completed business process mapping: 14

PAL Stage 1 Completed: 15

PAL Stage 2 Completed: 13

PAL Stage 3 Completed: 7

PAL Stage 4 Completed: 7

Completed Deployment of Modernized Software: 6

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2020

The Department (department) of Consumer Affairs' Business Modernization Plan of 2017 has effectively created a structured methodology for planning and implementing business process improvements and efficient online services to better serve the 3.2 million licensees and the department's many stakeholder groups. The plan has successfully delivered initial technology releases for 5 of 16 boards and bureaus (boards) in 2020 and has achieved many planning milestones for the remaining boards. The boards that have deployed modernized online services in 2020 are: the California Acupuncture Board, the Board of Chiropractic Examiners, the Board of Professional Engineers, Land Surveyors, and Geologists, Court Reporters Board, and the Bureau for Private Postsecondary Education. The chart below reflects the current progress.

Table 1. Phase 3 PAL and Business Activities Status

Phase 3 boards/bureaus with completed business process mapping: 12

PAL Stage 1 Completed: 14

PAL Stage 2 Completed: 7

PAL Stage 3 Completed: 7

PAL Stage 4 Completed: 7

Completed Deployment of Modernized Software: 5

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

DCA provided documentation demonstrating that several regulatory entities have completed Stage 2 of the PAL process, and provided documentation that it believes constitutes cost benefit analyses. However, it stated that there are nine regulatory entities that have not yet completed Stage 2 of the PAL process. Thus, the cost-benefit analyses for these remaining entities have not yet been finalized. Until such analyses are developed and approved, we will report this recommendation as partially implemented.

Annual Follow-Up Agency Response From November 2019

As reported in 2018, the Department of Consumer Affairs (DCA) has established a standard for planning, analyzing, documenting and justifying all reportable technological investments. Our standards, outlined in the department's 2017 Business Modernization Plan (updated in 2018.), require that the cost/benefit of an investment be analyzed and documented in Stage 2 of the Department of Technology's (CDT) Project Approval Lifecycle (PAL). Executive leadership from the board/bureau, DCA, the Business, Consumer Services and Housing Agency, the Department of Finance, and CDT then must approve the PAL documents validating the cost/benefit. The recommendation calls out specifically further investments in Breeze, but DCA has extended this requirement to all reportable technological investments.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

DCA provided documentation demonstrating that four regulatory entities have completed Stage 2 of the PAL process, and provided documentation that it believes constitutes cost benefit analyses. However, there are several regulatory entities that have not yet completed Stage 2 of the PAL process. Thus, the cost-benefit analyses for these remaining entities have not yet been finalized. Until such analyses are developed and approved, we will report this recommendation as partially implemented.

Annual Follow-Up Agency Response From October 2018

The Department of Consumer Affairs, in collaboration with phase 3 regulatory entities, has developed a Business Modernization Plan (and accompanying reports) to effectively facilitate the analysis, approval, and potential transition to a new licensing and enforcement platform. The Plan outlines our approach to identifying board/bureau specific business needs, determining cost-effective options, and proposes initial schedules. The Plan also incorporates the impacts and requirements of the Project Approval Lifecycle (PAL) - the project approval process required by the California Department of Technology. Each board or bureau will perform a cost- benefit analysis, or determine the cost effectiveness of each alternative platform during Stage 2 of the PAL process. The Financial Analysis Worksheets (FAW) facilitate fiscal planning and comparison and will be used to determine cost benefit on a board by board basis.

The phase 3 regulatory entities are in the following stages of PAL and business activities status: two entities have PAL Stage 1 completed, four entities for PAL Stage 1 in progress, two entities have PAL Stage 2 in progress, and nine entities have completed business process mapping.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From November 2017

The Department of Consumer Affairs, in conjunction with phase 3 regulatory entities, has developed a Business Modernization Plan (Plan) to effectively facilitate the analysis, approval, and potential transition to program specific licensing and enforcement platforms. The Plan outlines our approach to identifying board/bureau specific business needs, determining cost effective options, and proposes initial schedules. The Plan also considers the impacts and requirements of the Project Approval Lifecycle (PAL), the project approval process required by the California Department of Technology. Each board/bureau will perform a cost benefit analysis, or determine the cost effectiveness of each alternative platform, during Stage 2 of the PAL process. The Financial Analysis Worksheets (FAW) facilitate fiscal planning and comparison, and will be used to determine cost benefit on a board by board basis.

Currently there are four boards/bureaus actively documenting their current business processes to identify needs and requirements. The remaining boards/bureaus are working to inventory their business processes and schedule documentation workshops with departmental Organizational Change Management staff. One program has submitted Stage 1 of the PAL process and a second is scheduled to submit theirs within 90 days.

The thorough planning, business analysis, and program-specific nature of this effort will ensure success for our remaining boards and bureaus (formerly Release/Phase 3).

Implementation Timeframe: Proposed timelines are included and maintained in a board/bureau's individual Business Modernization Reports and may change due to business resource contention.

Individual board/bureau progress through business modernization will vary based on individual program resources, current legislative/business landscape, complexity, fiscal landscape, and the PAL process.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2016

The Department will perform a formal cost benefit analysis after Release 2 is completed. Part of this formal evaluation will include a gap analysis that will compare existing BreEZe functionality as delivered at the completion of Release 2, to Release 3 boards and bureaus' business needs and current systems' functionality to determine if BreEZe is a cost-effective IT solution for Release 3 boards. The results of this analysis will guide the Department's future efforts to implement BreEZe. In addition, the Department will ensure that the formal cost-benefit analysis is updated periodically as significant assumptions change.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2016

The Department will perform a formal cost benefit analysis after Release 2 is completed. Part of this formal evaluation will include a gap analysis that will compare existing BreEZe functionality as delivered at the completion of Release 2, to Release 3 boards and bureaus' business needs and current systems' functionality to determine if BreEZe is a cost-effective IT solution for Release 3 boards. The results of this analysis will guide the Department's future efforts to implement BreEZe. In addition, the Department will ensure that the formal cost-benefit analysis is updated periodically as significant assumptions change.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

The Department will perform a formal cost benefit analysis after Release 2 is completed. Part of this formal evaluation will include a gap analysis that will compare existing BreEZe functionality as delivered at the completion of Release 2, to Release 3 boards and bureaus' business needs and current systems' functionality to determine if BreEZe is a cost-effective IT solution for Release 3 boards. The results of this analysis will guide the Department's future efforts to implement BreEZe. In addition, the Department will ensure that the formal cost-benefit analysis is updated periodically as significant assumptions change.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The Department will perform a formal cost benefit analysis after Release 2 is completed. Part of this formal evaluation will include a gap analysis that will compare all existing BreEZe functionality as delivered at the completion of Release 2, to the Release 3 boards and bureaus' business needs and current systems' functionality to determine if BreEZe would be the most cost-effective IT solution for Release 3 boards. The results of this analysis will guide the Department's future efforts to implement BreEZe. In addition, the Department will ensure that the formal cost-benefit analysis is updated periodically as significant assumptions change.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

The Department will perform a formal cost benefit analysis after Release 2 is completed. Part of this formal evaluation will include a gap analysis that will compare all existing BreEZe functionality as delivered at the completion of Release 2, to the Release 3 boards and bureaus' business needs and current systems' functionality to identify necessary BreEZe enhancements. The results of this analysis will guide the Department's future efforts to implement BreEZe. In addition, the Department will ensure that the formal cost-benefit analysis is updated periodically as significant assumptions change.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2014-116

Agency responses received are posted verbatim.