2020-041 - Status of Recommendations - Table 2

7. To ensure that BRN resolves complaints regarding nurses in a timely manner, by March 1, 2017, it should establish a plan to eliminate its backlog of complaints awaiting assignment to an investigator.

10. To increase its pool of expert witnesses, by June 2017, BRN should take the steps necessary to increase the hourly wage it pays expert witnesses.

17. To the extent that Consumer Affairs chooses to implement BreEZe at the phase 3 regulatory entities, it should first complete a formal cost-benefit analysis to ensure that BreEZe is a cost-effective solution to meet these regulatory entities' business needs. To make certain this analysis is complete, it should include an assessment of the potential changes these regulatory entities may require to be made of the BreEZe system and the associated costs. Consumer Affairs should complete the cost-benefit analysis before investing any more resources into the implementation of BreEZe at the phase 3 regulatory entities, and it should update this analysis periodically as significant assumptions change.

35. To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should provide training on the BreEZe system as close to the rollout date as possible to ensure that staff retain the information for using the system as it is implemented.

36. To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should work with the regulatory entities to develop training that is specific to each entity's business processes.

5. To ensure that it appropriately monitors CalHome as required by statute, regulation, and program guidelines, HCD should immediately collect all required reports and follow up with recipients to obtain missing reports. Staff should withhold fund disbursements from recipients that have not submitted required reports. If the submitted reports reveal a problematic trend, such as a recipient not disbursing funds, HCD should take appropriate corrective action with the recipient.

8. To ensure that the IIG program award funds benefit the target population, HCD should develop and use a tool by December 1, 2018, to track which awards are monitored by local jurisdictions or by other HCD programs and which are not monitored at all. HCD should then immediately obtain monitoring reports from the local jurisdictions and other HCD programs to verify monitoring and review the results of such monitoring. HCD should follow up on any noted deficiencies. Further, HCD should, by January 1, 2019, develop a plan to perform on-site visits for those recipients that do not receive adequate monitoring from another source, and it should perform the planned on-site monitoring.

14. To ensure that it maximizes the benefit of the funds it has invested in CAPES's development and to support its ongoing efforts to improve CAPES's usability, HCD should, by January 1, 2019, develop a documented process to ensure that all data in CAPES are accurate and complete. This process should include all phases of contract management, including monitoring. HCD should implement a routine periodic review of this process and update the process as necessary.

16. To ensure that it is able to meet its administrative monitoring obligations and that it uses housing bond funds in compliance with state law, regulations, and program guidelines, HCD should develop a long-term plan by January 1, 2019, for how it will avoid exceeding the administrative cost limits of those programs in the most immediate danger of overage and for how it will address instances when it has exceeded administrative cost limits. The plan should identify the programs at risk of exceeding the limit; the actions HCD will take for each program to gain efficiencies; its plan for moving staff between programs; a request for more money or legislative changes such as modifying the statutory limit on administrative spending, if necessary; and an evaluation of the consequences of not fulfilling its monitoring obligations.

18. To ensure that it complies with state law, prudently uses administrative funding, and promotes transparency, HCD should calculate and retain only funds equal to its actual administrative costs in instances when it does not disburse awarded funds to a recipient and subsequently grants the funds to another recipient.

19. To ensure that it does not exceed administrative cost restrictions and that it maximizes the funds intended to address target populations' housing needs, HCD should estimate when it will run out of administrative funds for any specific program, document its projection methodology, and provide underlying data and support for its estimates. The projections should include, but not be limited to, actual staff time spent on the program, the number of awards being monitored, and the length of monitoring. Staff should provide these projections and methodologies to management for review and approval by December 1, 2018, and then at least biannually thereafter.

35. Following the completion of the analysis (described in Recommendation 34), the Medical Board should take the appropriate follow-up actions that it deems necessary, including the investigation of physicians identified in its analysis.

14. Ensure that scheduling staff use the appropriate post codes in the scheduling software for all nursing staff so their time is accurately reported.

15. Ensure that all other nursing staff assignments to ad hoc posts are appropriate, and require schedulers to enter a note in the scheduling software indicating an employee's duties while in a general ad hoc post.

2. Corrections should immediately require mental health staff to score 100 percent on risk evaluation audits in order to pass. If a staff member does not pass, Corrections should require the prison to follow its current policies by reviewing additional risk evaluations to determine whether the staff member needs to undergo additional mentoring.

15. To ensure that it has enough staff to provide mental health services to all inmates who require care, Corrections should review and revise its mental health staffing model by August 2018.

16. To ensure that prisons comply with its policies related to suicide prevention and response, Corrections should continue to develop its audit process and implement it at all prisons by February 2018. The process should include, but not be limited to, audits of the quality of prisons' risk evaluations and treatment plans.

17. To ensure that prisons can easily access Corrections' current policies related to mental health, Corrections should ensure that its program guide is current and complete as it works to incorporate the program guide into regulations. Corrections should immediately begin working with federal court monitors to draft regulations.

18. To ensure that suicide prevention teams meet quorum requirements, Corrections should, starting January 2018, work with prisons that consistently fail to achieve a quorum to resolve issues that may be preventing the teams from having all required members present at meetings.

1. To ensure that Corrections has reliable tools for assessing the needs of its inmate population, it should validate COMPAS and CSRA by January 2020 and revalidate all of its assessment tools at least every five years.

2. To ensure that Corrections is able to discover and prioritize the most effective CBT rehabilitation curricula, it should begin using its ability to record the individual CBT curricula inmates receive, and then use this information in an analysis of its rehabilitation programs in 2020.

8. To increase the space available for rehabilitation programs, by January 2020 Corrections should analyze and report on its current infrastructure capacity compared to its needs for the programs. The report should include the current space available and the square footage needed. If the report indicates that additional space is necessary, Corrections should work with the Legislature to address those needs.

9. To improve the inmate enrollment rates in CalPIA's vocational education programs, CalPIA and Corrections should require a CalPIA representative to attend all classification committee meetings at all nine prisons where CalPIA offers vocational education. Corrections should also ensure that it enrolls eligible inmates in CalPIA's vocational programs before filling spots in its own vocational programs. In addition, if the CalPIA recidivism study indicates that CalPIA's vocational programs are better at reducing recidivism than Corrections' vocational programs, CalPIA should request funding from the Legislature to expand its vocational training program.

11. To ensure that Corrections effectively and efficiently allocates resources and reduces recidivism, it should partner with a research organization to conduct a systematic evaluation during fiscal year 2020-21 to determine whether its rehabilitation programs are reducing recidivism and if they are cost-effective. In addition, the external researcher should provide input on the development of performance targets, including recidivism reduction. Depending upon the results of the analysis, Corrections should then eliminate or modify programs that prove ineffective.

12. To ensure that Corrections effectively and efficiently allocates resources and reduces recidivism, it should partner with an external researcher during fiscal year 2020-21 to help it quantify the effect volunteer programs have on inmate outcomes and consider expanding those programs if they prove effective or ceasing them if they are not effective.

13. To ensure that Corrections effectively and efficiently allocates resources and reduces recidivism, it should collaborate with C-ROB during fiscal year 2019-20 to establish annual targets for reducing recidivism and determining the cost-effectiveness of the programs. Corrections should also request federal grants tied to setting targets for recidivism reduction.

21. To ensure the health and safety of its officers when interacting with inmates, CIM should provide annual training that is specific to preventing and responding to gassing attacks.

15. CDCR should revise the Institutional Worker Supervision Pay (IWSP) procedure to require that personnel staff review and ensure that an employee's direct supervisor signs the qualifying employee's timesheets and IWSP documents each month.

17. CDCR should enforce its current procedure to retain Institutional Worker Supervision Pay (IWSP) documentation.

19. CDCR should train all employees, supervisors, and personnel staff who receive, approve, or issue the extra pay to ensure that they are familiar with the requirements of the IWSP procedure and Pay Differential 67.

20. Fully implement and continue to follow recommendations from prior investigative reports involving similar inappropriate inmate supervision pay at other CDCR facilities, including the recommendation to train all employees who receive, approve, or issue the inmate supervision pay.

10. To improve the inmate enrollment rates in CalPIA's vocational education programs, CalPIA and Corrections should require a CalPIA representative to attend all classification committee meetings at all nine prisons where CalPIA offers vocational education. Corrections should also ensure that it enrolls eligible inmates in CalPIA's vocational programs before filling spots in its own vocational programs. In addition, if the CalPIA recidivism study indicates that CalPIA's vocational programs are better at reducing recidivism than Corrections' vocational programs, CalPIA should request funding from the Legislature to expand its vocational training program.

16. To ensure that Corrections is taking steps to reduce recidivism, C-ROB should monitor whether Corrections is developing appropriate recidivism targets and, in its annual report, should evaluate Corrections' progress toward meeting those targets.

3. To comply with state laws and regulations and help ensure that staff involved in making governmental decisions during the grant process are impartial, Rehabilitation should ensure that they receive ethics training, which includes conflict-of-interest training, at least every two years.

6. To increase transparency and ensure that applicants have the information necessary to understand the grant process, Rehabilitation should include in its RFAs clear scoring criteria and descriptions of the evaluation, award, and appeals processes, including the process it will use to address applications that receive tied scores.

11. To increase the transparency of its selection process and to ensure that it receives the most qualified evaluators possible, Rehabilitation should issue a public solicitation for evaluators for each grant that includes a description of essential and desirable qualifications.

13. To ensure that it provides sufficient oversight of the grant process, Rehabilitation should ensure that the technical review teams it assigns to grants provide the director and chief deputy with a memorandum summarizing the evaluation process and the evaluators' recommended grant awardees. Rehabilitation should also designate an individual responsible for reviewing and approving the memorandum and recommended awardees before it publishes its notice of intent to award.

14. If it finds errors in an evaluation that merit restarting the grant process, rescoring of applications, or convening a new evaluation panel, Rehabilitation should resolve any issues before it begins the rescoring process. It should also notify applicants to ensure that they are aware of any changes to the process due to the errors. Further, it should consider promulgating regulations and amending its grant manual to permit staff to request evaluators to rescore applications or convene a new evaluation panel when it finds issues with an evaluation.

15. To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual that staff at the appropriate level of authority are to acknowledge all appeal requests, notify intended awardees that could be affected by the appeals, and inform the appellant of the qualifications of the review committee members. Staff at the appropriate level of authority must also notify all affected parties of the review committee's final decision within the time frame Rehabilitation establishes in regulations.

16. To ensure that Rehabilitation has appropriate oversight of its grant process and can sufficiently demonstrate that it followed the process, it should designate staff, separate from those involved in the respective grant process, to conduct a review of each grant process for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications before it awards grants.

17. To comply with federal and state requirements, and to ensure consistency and fairness in its grant process, Rehabilitation should revise and formalize the policies and procedures in its grant manual to incorporate the rules adopted by regulation and to address the recommendations in this report. The grant manual should specify that any deviations from the required grant process must be for good cause and be documented.

18. To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual a process for the review committees to request additional information from appellants or program staff. To allow time for an adequate review of any additional information, Rehabilitation should consider extending the time for review committees to issue their decision on appeals from 30 days to 45 days.

19. To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual that to be able to rescore applications when necessary, the review committee members should be subject-matter experts or, if they are not subject-matter experts, the review committee should have the authority to recommend a new evaluation panel instead of rescoring applications itself when it identifies a reason to invalidate previous evaluations.

5. In order to bring violators of the recycling act into compliance and to ensure that its enforcement activities are timely, CalRecycle should do the following:

- Assess penalties for noncompliance with the recycling act.

- Publicize any penalties it assesses against violators of the recycling act as a deterrent to potential violators.

- Monitor inspection cases to ensure that it does not complete them before the retailers in question have remedied any instances of noncompliance.

- Execute a plan to verify compliance for all inspections in which it did not obtain evidence of compliance.

- Develop and implement a timeline for the penalty phase of the enforcement process.

- Regularly review the timeliness of its enforcement process and prioritize any overdue enforcement actions based on its enforcement timelines.

2. To ensure it can demonstrate that its fraud prevention efforts are maximizing financial recoveries for the beverage program, CalRecycle should both modify and annually update its fraud management plan to include the following:
- By December 31, 2014, formally establish a systematic process for analyzing, monitoring, and responding to the risk of fraudulent recycling of out-of-state beverage containers.
- Develop fraud estimates—by type of fraudulent activity—that quantify the potential financial losses to the beverage program and the methodology CalRecycle used to develop these estimates.
- Identify the amount of actual fraud in the prior year by type of fraudulent activity, such as the financial losses resulting from the redemption of out-of-state beverage containers or the falsification of reports used to substantiate program payments.
- Identify the amount actually recovered for the beverage program in the form of cash for restitution and penalties resulting from fraud.

3. To allow for public input and to prevent any legal challenges claiming that its policies and procedures regarding prepayment holds constitute unenforceable underground regulations, CalRecycle should adopt these policies and procedures as regulations in accordance with the Administrative Procedure Act.

1. Take appropriate corrective action against the district engineer and the supervisors for their participation in or failure to address the conflict of interest.

7. If the State Water Board believes regulations are necessary to ensure that the regional boards and local jurisdictions follow its guidance regarding adequate and consistent information pertaining to their costs for storm water management, the State Water Board should adopt such regulations.

11. The State Water Board should revise its trash policy to focus it on local jurisdictions that have water bodies that are harmed by trash, as identified by the polluted waters list. In addition, the State Water Board should review the polluted waters list at least biannually to identify any additional water bodies recently determined to be harmed by trash and impose its trash policy on the applicable jurisdictions.

1. CDFA:
Ensure that its audit office conducts biannual compliance audits for all district agricultural associations and that the office prioritizes auditing district agricultural associations on the watch program.

3. CDFA:
Provide district agricultural associations with timely notifications from the Department of Motor Vehicles' Employer Pull Notice Program and follow up to ensure that the district agricultural associations take appropriate action.

4. CDFA:
To the extent that its authority allows, oversee implementation of our recommendations to the association.

5. Association:
Take appropriate disciplinary action against the CEO, the maintenance supervisor, and all other permanent and temporary employees who engaged in the improper governmental activities that we identified.

12. Association:
Formally adopt, train staff on, and follow the Fairs and Expositions branch's accounting procedures manual, as well as CDFA's Controlled Substances and Alcohol and Incompatible Activities policies.

10. To ensure that the veterans homes receive all of the funding to which they are entitled, by the May 2019 budget revision, CalVet should seek an augmentation to its appropriation for the homes equal to the lease revenues it generated from July 2015 through June 2018. If CalVet believes the state law requiring lease proceeds to augment its appropriation is outdated, it should seek a change to state law.

11. To monitor whether lessees are current on payments, CalVet should track payment compliance for all lease payments that it receives and promptly follow up with lessees that do not pay as required. This should include collecting sufficient records from lessees that pay rent based on a percentage of sales to calculate the amount that each is required to pay to ensure that the lessees are making the correct rent payments.

16. To prevent unauthorized use of its property, CalVet should regularly monitor the use of the leased properties and take action to cease any activity that is not allowed by the terms of the lease agreements. Further, it should take action to cease the balloon launches from the golf course or amend its lease with the lessee to identify balloon launches as an approved use of the property.

9. To ensure that its outreach efforts are effective and result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking, CalVet should assess, at least annually, the effectiveness of its past outreach efforts in increasing the number of DVBE firms that become certified.

11. To ensure that its outreach efforts are effective and result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking, CalVet should work with awarding departments to identify the types of goods and services for which they struggle to find a DVBE contractor or subcontractor.

13. To ensure that its outreach efforts are effective and result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking, CalVet should develop an outreach plan to include outreach activities found to be effective in the past based on its assessment. This plan should also emphasize outreach to increase the number of DVBE firms that provide the types of goods and services that awarding departments struggle to obtain from DVBE firms.

15. To ensure that its outreach efforts are effective and result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking, CalVet should develop better tracking of the businesses owned by disabled veterans that attend its outreach events and review certification data to determine whether these businesses obtained their certifications.

17. To ensure that its outreach efforts are effective and result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking, CalVet should conduct periodic surveys of businesses owned by disabled veterans that attended its outreach events but chose not to become DVBE certified to determine the reasons for not applying for the certification. The department should use this information to improve its outreach and any other areas of the program.

19. Until the Legislature amends the law to transfer its responsibility for assisting underachieving departments to General Services, CalVet should develop and follow policies and procedures to identify and assist awarding departments that fail to meet, or are at risk of not meeting, the 3 percent DVBE participation goal.

9. To prevent unnecessary delays and use of resources and to ensure its compliance with state law, the commission should, following the Legislature's amendment of the Gambling Act that we recommend, revise its regulations and policies for conducting evidentiary hearings. These revisions should specify that the commission may vote at regular meetings on a final basis to approve or deny licenses, registrations, permits, findings of suitability, or other matters and that it is not required to conduct evidentiary hearings unless applicants request that it do so.

13. To better align the revenue in the Gambling Fund with the costs of the activities that the fund supports, the bureau and the commission should conduct cost analyses of those activities by July 2020. At a minimum, these cost analyses should include the following:

The entities' personnel costs, operating costs, and any program overhead costs.

Updated time estimates for their core and support activities, such as background investigations.

The cost of their enforcement activities.

Using this information, the bureau and commission should reset their regulatory fees to reflect their actual costs. Before conducting its fee study, the bureau should implement our recommendations to improve its processes for assigning applications, ensuring the completeness of applications, and developing time-reporting protocols.

6. To ensure that the CPUC fulfills its statutory requirement for auditing all water utilities, it should immediately begin to follow its Standard Practice when auditing Class A water utilities, or develop policies and procedures by May 2019 to ensure that the reviews Public Advocates conducts of general rate cases demonstrate compliance with the legal requirement for audits of these utilities.

7. To ensure that the CPUC fulfills its statutory requirement for auditing all water utilities, it should immediately develop a plan to complete audits of Class A water utilities and small water utilities in a timely manner.

3. To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in CIMS, the branch should continue to implement its quality management team program component focused on reviewing the categorization of complaints and correcting identified errors.

4. To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in CIMS, the branch should develop and implement tools by September 30, 2015, to measure the quality management team program's effectiveness.

11. The commission should ensure that it complies with all policy requirements in SAM Chapter 5300 no later than April 2016.

16. The commission should revise its existing recovery plan to include a list of applications supporting critical business functions, their maximum acceptable outage time frames, and detailed recovery strategies for each application.

17. The commission should revise its existing recovery plan to include detailed procedures for rebuilding its technology infrastructure at an alternate processing site.

18. The commission should conduct regular tests and exercises to assess the sufficiency of the revised recovery plan and refine the plan when necessary.

2. To ensure that the choice of a vendor is sufficiently justified and that the vendor represents the best value, the CPUC should explain in its final decision how the vendor was the most qualified in all cases when the CPUC does not competitively select the vendor it directs utilities to contract with.

4. To avoid the appearance of inappropriate relationships, the CPUC should adopt a policy to prohibit commissioners from accepting gifts from regulated utilities and energy companies and free travel from organizations with significant ties to regulated utilities and other parties with financial interests in CPUC proceedings.

8. To ensure that its contracting practices align with state requirements and best practices, the CPUC should designate a limited number of project managers for each division at the CPUC, and provide those individuals with training on the CPUC's processes related to contracting, including how to monitor progress of a contractor's work.

13. The CPUC should update its regulations to require parties joining a proceeding by filing a protest or response to an application or petition, or by filing comments in response to a rulemaking proceeding to fully disclose their interests in the proceeding.

3. To show how air pollution emissions reductions related to the solar initiative benefit the State, the commission should include in future reports the measurable benefits of those reductions.

14. To ensure that IT projects have the oversight needed to better position them for success, CalTech should develop thresholds relating to IT project cost increases and schedule delays to inform and better justify its decision to allow an IT project to continue. If a department's IT project reaches or exceeds these thresholds, CalTech should require the department to conduct a cost-benefit analysis for the project and include this analysis in an SPR. CalTech should consider the results of this analysis in its decision to approve or deny the SPR and, if warranted, take action to suspend or terminate the project so that it does not allow projects with significant problems to continue without correction.

2. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should ensure that the project management system can centrally track and extract all data regarding project status, including time delays, cost overages, and the reasons for each.

3. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should track the reasons that projects are pending to identify its true backlog of projects. In doing so, it should develop a process to follow up on those projects that are pending to ensure that they are not on hold unnecessarily and are appropriately moving forward.

4. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should, at least annually, use the centrally tracked data to identify common themes in the causes for project delays and cost overages and develop solutions to address these issues. Further, it should report the results of its review to General Services' executive management.

5. Until the division implements its planned project management system, it should, by September 2016, develop a process to, at a minimum, identify project status and reasons for project delays as well as cost overages. Using these data, the division should modify its project management processes to ensure the efficient and effective delivery of projects.

6. The division should develop and implement a process for preparing reasonable time frames and cost estimates for its projects within the building management branch. To better inform the development of this process, the division should evaluate the branch's structure, which should include a staffing analysis, to determine whether it is effectively organized and whether it should add cost estimator positions.

7. To ensure that client agencies are paying equitable rates, by December 2016 General Services should develop and implement a strategy for allocating its administrative costs equally among all the projects it completes for client agencies, including those portions outsourced to private firms.

8. To ensure that the project management branch charges its client agencies a competitive hourly rate, by December 2016 and every two years thereafter, the division should conduct a rate analysis that fully accounts for differences between the project management branch's rate and private firms' rates. If it finds that the rates are not competitive, the division should identify and implement strategies to ensure that the project management branch's rates are as competitive as they can be with those of its private firm counterparts. Further, the division should explore and implement any other reasonable methods to ensure that it is delivering projects as cost effectively as possible.

10. To improve its communication with client agencies, the division should develop a process for providing periodic detailed bills and invoices to client agencies clearly describing the work for which it is charging.

11. To effectively evaluate the performance of its branches in delivering projects, the division should develop meaningful goals and objectives and a method of measuring its success in achieving them as part of its strategic plan that is focused on ensuring that projects are delivered on time and within budgeted cost estimates.

12. To ensure that its project management staff are adequately trained and have the information necessary to deliver projects as efficiently and effectively as possible, the division should conduct a comprehensive survey every other year of all of its client agencies to inform necessary improvements to its processes and training program and, in the interest of transparency, make the survey results public.

3. To improve its oversight of the State's noncompetitive contracts, General Services should, within 90 days, create plans for regularly performing statewide analyses to identify potential abuse or overuse of noncompetitive contracts. These analyses should include, but not be limited to, calculating the proportional value and number of the State's competitive and noncompetitive contracts and amendments, examining trends in agencies' use of noncompetitive contracts and amendments, and identifying unusual patterns among vendors receiving state contracts through noncompetitive means.

2. To increase its efforts to prevent and control diabetes, Public Health should develop a process for identifying and applying for federal funding opportunities, including routinely and proactively searching for grants. In addition, Public Health should seek funding for a grants specialist position to identify and apply for federal and other grants.

8. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should address staffing issues by preparing and resubmitting to Public Health a recruitment and retention proposal, developing a succession plan, and taking necessary steps to implement its planned reorganization.

9. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should ensure that its information technology data systems have necessary safeguards, contain accurate and complete data, and support its program needs.

10. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should update and develop its regulations as necessary to ensure consistency with existing state law.

8. As the Legislature considers changes to state law, Public Health should take the steps necessary to ensure that its oversight results in nursing facilities improving their quality of care by amending its application licensing reviews by developing a defined process that specifies how an analyst will determine whether an applicant has demonstrated its ability to comply with state and federal requirements. This process also needs to ensure that analysts conduct complete and standardized reviews of each nursing facility application within each class of facility. Specifically, these processes should clearly outline what factors analysts will consider when determining whether an applicant is in compliance, how analysts will weigh those factors for each class of facility, and what objective thresholds will prompt analysts to elevate applications for review and approval by higher-level management. Additionally, Public Health should document the additional factors higher-level management will consider if applications are elevated for their review to ensure that Public Health conducts standardized reviews of nursing facility applications of the same class. Finally, Public Health should develop processes ensuring that it documents its decisions adequately.

9. As the Legislature considers changes to state law, Public Health should take the steps necessary to ensure that its oversight results in nursing facilities improving their quality of care by ensuring that it issues citations in a timely manner, especially for immediate jeopardy deficiencies.

11. To improve the availability and transparency of information, Public Health should upload all inspection findings to Cal Health Find and review ownership data by May 2019.

10. To ensure that Social Services evaluates the risk individuals may pose to vulnerable populations in its licensed care facilities as quickly as possible, by July 2017 Social Services should establish time frames for staff to evaluate individuals who are present in their facilities and who have received administrative actions from other departments. In addition, it should monitor and follow up with the appropriate staff regarding the status of their assessments of these individuals and their final decisions.

13. To comply with state law and better protect vulnerable populations in California's licensed care facilities, Social Services should immediately change its policy to require that its exemption analysts evaluate all infraction convictions, other than minor traffic violations, before granting exemptions to individuals. If Social Services believes it is not feasible to evaluate all of these convictions, it should report to the Legislature by June 2017 how it ensures that vulnerable populations are not at risk and should request that the Legislature change the law to eliminate infraction convictions as a crime category that Social Services must evaluate in order to grant an exemption.

16. Until the Legislature requires that Social Services receive both California and federal criminal history information before issuing a clearance or processing an exemption, to better protect vulnerable populations, Social Services should immediately revise its policy to require its regional offices to obtain all self-disclosure forms for individuals who submit fingerprints to Justice as part of an application to be present in a licensed facility. The regional offices should then forward to the CBCB all self-disclosure forms that identify a conviction.

17. Until the Legislature requires that Social Services receive both California and federal criminal history information before issuing a clearance or processing an exemption, to better protect vulnerable populations, Social Services should immediately change its practice of allowing individuals who have not submitted a self-disclosure form to Social Services to have access to licensed facilities, thus reflecting the requirements of state law. In addition, the CBCB should develop a process to ensure that individuals cannot receive a clearance or an exemption without the CBCB first receiving both California and federal criminal history information if a regional office does not have a self-disclosure form for the individual.

26. To ensure that regional offices pursue legal actions in a timely manner, by July 2017 Social Services' headquarters should identify a resource—such as a unit—to monitor and follow up with the regional offices regarding the status of their legal actions related to substantiated address matches of registered sex offenders at licensed facilities.

30. To improve the oversight of psychotropic medications prescribed to foster children, Social Services should collaborate with the counties and other relevant stakeholders—including Health Care Services, as necessary—to develop and implement a reasonable oversight structure that ensures the coordination of the State's and counties' various oversight mechanisms as well as the accuracy and completeness of the information in Social Services' data system.

4. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should require counties to give licensed foster homes a higher priority than foster family agencies for children that do not have identified treatment needs.

5. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should require counties to prepare a detailed justification for any child placed with a foster family agency.

5. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should, using the results from the recommended cost-effectiveness analysis, determine why some counties' efforts to combat welfare fraud are more cost-effective than others.

6. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should seek to replicate the most cost-effective practices among all counties. Social Services should work with its legal counsel to determine whether to withhold information about these practices from public disclosure.

7. Social Services should track counties' prosecution thresholds for welfare fraud cases and determine whether they affect counties' decisions to investigate potential fraud, with a focus on determining best practices and cost-effective thresholds. If Social Services' analysis determines that varying prosecution thresholds do affect counties' decisions, it should then work with counties to implement the consistent use of these cost-effective prosecution thresholds.

9. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should address and promptly act on the four remaining recommendations that its steering committee provided in 2008.

12. To make counties' review of match lists more efficient, Social Services should revive its efforts to work with the state and federal agencies that prepare the match lists to address the counties' concerns about match list formats, content, and criteria.

2. To ensure timelier fee assessments, Developmental Services should hold regional centers accountable for providing the monthly placement reports and copies of information letters required by state regulations. To encourage compliance, Developmental Services should specify in its regional center contracts that noncompliant regional centers will pay financial penalties equal to the amount of revenue lost because of their inaction.

4. To ensure that regional centers are aware of the benefits, including cost savings to the State that can be realized by using FMS vendors, DDS should formally communicate to regional centers regarding the model.

5. To ensure that in-home respite vendors are providing quality services and that vendors are adhering to state requirements, DDS should issue regulations requiring regional centers to conduct periodic and ongoing reviews of vendors' programs, employees, and consumer records.

6. To ensure that in-home respite vendors comply with vendor requirements on an ongoing basis, DDS should require the regional centers to develop a process to conduct biennial reviews of the vendor files the regional centers maintain and document the outcome of the review in the files. DDS should require the regional centers to take appropriate action to ensure that vendors comply, up to and including terminating the vendorization, if necessary.

7. To ensure that it is providing oversight in accordance with state law and federal requirements, DDS should ensure that it performs audits of each regional center every two years as required. In conducting these audits, DDS should consistently include a review of in-home respite services.

6. If Health Care Services finds significant errors in a health plan's provider directory, it should work with that health plan to identify reasons for the inaccuracies and require the health plan to develop processes to eliminate the inaccuracies.

1. To ensure that it provides claiming units with reasonable opportunities to address concerns with its decisions or actions, Health Care Services should, within three months, begin preparing regulations to establish and implement a formal appeals process that allows claiming units to directly appeal Health Care Services' decisions.

2. To ensure that it provides claiming units with reasonable opportunities to address concerns with its decisions or actions, Health Care Services should, within three months, inform all stakeholders, including claiming units, of the existence of this appeals process.

4. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should complete the oversight reviews for at least three high-risk local educational consortia or local governmental agencies by December 31, 2015, and post the results to its website.

5. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should complete the oversight reviews for any remaining high-risk local educational consortia or local governmental agencies by June 30, 2016, and post the results to its website.

8. To minimize the risk that claiming units could include unallowable costs when calculating their reimbursement claims, Health Care Services should remind all local educational consortia and local governmental agencies that contracts with their claiming units should prohibit claiming units from seeking federal reimbursement of Health Care Services' participation fee.

10. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Heath Care Services should implement a single statewide quarterly random moment time survey and develop and implement a plan to take over responsibility for conducting quarterly time surveys and performing related activities as soon as reasonably possible.

11. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Heath Care Services should implement a single statewide quarterly random moment time survey and develop and issue a request for proposals to identify a responsible vendor to assist in implementing a statewide quarterly random moment time survey.

12. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Heath Care Services should implement a single statewide quarterly random moment time survey and draft revisions to regulations as appropriate and to applicable documents, including the manual, oversight strategies and plans, and policy and procedure letters.

13. To the extent that local educational consortia and local governmental agencies are no longer involved in the administrative activities program, Health Care Services should develop and issue a standard contract for claiming units to sign to participate in the program.

17. To better maximize federal reimbursements for the administrative activities program, Health Care Services should, within six months, develop and implement a method to oversee and track the outreach efforts that local educational consortia and local governmental agencies use for ensuring that nonparticipating claiming units understand the benefits and consider participating in the administrative activities program.

21. To provide the public with the ability to participate fully in developing the rules governing the administrative activities program, Health Care Services should, in accordance with California's Administrative Procedure Act (APA), immediately develop and adopt the regulations cited in the four subdivisions of Section 14132.47 of the California Welfare and Institutions Code.

20. To make certain that it meets the requirements of the new state law and that its performance measures are accurate, Health Care Services should establish the provider-to-beneficiary ratio statewide and by county as performance measures designed to evaluate access and availability of dental services and include this measure in its October 2015 report to the Legislature.

23. To ensure that Health Care Services and its fiscal intermediaries reimburse providers only for services rendered to eligible beneficiaries, Health Care Services should obtain Social Security's Death Master File and update monthly its beneficiary eligibility system with death information.

4. By September 2019, and periodically thereafter, DHCS should conduct another risk assessment and ensure that it includes a comprehensive evaluation of which contract areas—including conflicts of interest—it should focus on in its annual medical audits.

5. Going forward, DHCS should conduct a comprehensive risk assessment and ensure that it reviews health plans' conflict-of-interest controls at least once every three years.

6. DHCS should develop and issue an All-Plan letter or other binding guidance by March 2020 to the health plans that specifically defines what constitutes reasonable and necessary administrative expenses.

7. DHCS should provide guidance to health plans on what is a reasonable bonus program. In doing so, DHCS should perform the necessary oversight to ensure health plans comply with this direction.

1. To ensure that beneficiaries in Regional Model counties have adequate access to care, DHCS should identify by August 2020 the locations requiring additional providers and the types of providers required. It should also develop strategies for recruiting and retaining providers in those locations. If it requires additional funding to complete this assessment or to implement actions to address its findings, DHCS should determine the amounts it needs and request that funding from the Legislature.

10. To ensure that all counties are aware of the managed care model options available to them and of the steps necessary to implement those models, DHCS should provide by December 2019 information to all counties that clearly defines each managed care model and the steps and legal requirements needed to establish each model.

11. To ensure that it makes informed decisions regarding the extension or renewal of its contracts with managed care health plans, DHCS should immediately begin the practice of requesting annual feedback from the counties that the health plans serve and of using that feedback in its decision-making process.

13. To ensure that beneficiaries in the Regional Model counties have reasonable access to care, DHCS should evaluate by June 2020 whether the structural characteristics of a COHS Model would be better suited to providing reasonable access to care in the Regional Model counties and notify the counties whether a COHS would improve beneficiaries' access to care. If some or all of these counties desire to transition to a COHS, DHCS should assist them in making that change after their current contracts expire.

14. To ensure that beneficiaries in the Regional Model counties have reasonable access to care, DHCS should evaluate by June 2020 whether it has the financial resources to provide assistance to counties interested in establishing a COHS or other managed care model after the current Regional Model contracts expire. If DHCS does not have the required financial resources, it should seek an appropriate amount of funding from the Legislature.

15. To ensure that beneficiaries in the Regional Model counties have reasonable access to care, DHCS should provide these counties by June 2020 with reasonable opportunities to decide whether to change their managed care models after the expiration of the Regional Model health plan contracts. DHCS should provide counties that choose to do so sufficient time to establish their new models. DHCS should also include language in its 2020 RFP to allow Regional Model counties that can demonstrate their ability to implement a COHS Model in their county by 2023 to opt out of the RFP process.

2. To recover inappropriately spent funds, prevent future erroneous payments, and ensure eligible individuals' access to care, Health Care Services should resolve the discrepancies we identified and recover erroneous payments where allowable by June 30, 2019.

3. To prevent future erroneous payments, Health Care Services should implement procedures by December 31, 2018, to ensure the timely resolution of system discrepancies. These procedures should include Health Care Services regularly following up on recurring, unresolved system discrepancies with the responsible county.

4. To prevent future erroneous payments, Health Care Services should establish procedures by December 31, 2018, that define when it will use its authority as defined in state law to sanction unresponsive counties that do not remedy known discrepancies.

6. To assist counties in addressing discrepancies, Health Care Services should reevaluate and update its guidance to the counties related to prioritizing MEDS alerts by December 31, 2018.

6. To increase access to preventive health services for children in areas where they are needed most, DHCS should identify by September 2019 where more providers who see children are needed and propose to the Legislature funding increases to recruit more providers in these areas.

7. To ensure that children in Medi-Cal have access to all of the preventive services for which they are eligible, DHCS should modify by May 2019 its contracts to make it clear to plans and providers that they are required to provide services according to Bright Futures.

8. To ensure that eligible children and their families know about all the preventive services they are entitled to through Medi-Cal, DHCS should include by May 2019 clearer and more comprehensive information about those services in its written materials and by September 2019 ensure annual follow-up with any children and their families who have not used those services.

9. To improve access and utilization rates, DHCS should establish by March 2020 performance measures that cover Bright Futures services through well-child visits for all age groups, and require plans to track and report the utilization rates on those measures.

11. To ensure that plans address underutilization of children's preventive services, DHCS should require plans by September 2019 to use their utilization management programs to identify barriers to usage specifically for these services and hold the plans accountable to address the barriers they identify.

12. To better ensure the accuracy of its data and ensure that California receives all available federal Medicaid funding, DHCS should require its EQRO to perform its encounter data validation studies annually using the most recent set of data available, and it should implement recommendations from its EQRO studies.

13. To ensure that plan provider directories are accurate, by September 2019 DHCS should begin using a 95 percent confidence level and not more than a 10 percent margin of error on its statistical sampling tool and should require at least 95 percent accuracy before approving a plan's provider directory. In addition, DHCS should ensure that its staff adhere to its policy to retain all documentation related to its review of provider directories for at least three years.

14. To mitigate health disparities for children of differing ethnic backgrounds and language needs, DHCS should revise by September 2019 the methodology for its EQRO's health disparity study to enable it to better make demographic comparisons, and it should use the findings to drive targeted interventions within plan service areas. It should publish this study annually.

16. To help increase utilization rates, DHCS should begin by September 2019 to monitor and identify effective incentive programs at the plan level and share the results with all plans.

18. To improve its ability to ensure that children are receiving recommended preventive health services, DHCS should create by September 2019 an action plan to annually address the EQRO's recommendations relating to children's preventive services, including recommendations left unaddressed from the previous two years' reports.

6. To ensure that local mental health agencies appropriately spend MHSA funds, Health Care Services should publish its proposed regulations in the California Regulatory Notice Register by September 2018. It should then develop and implement an MHSA fiscal audit process, independent of the Medi-Cal reviews, to review revenues and expenditures for the most recent fiscal year.

12. Health Care Services should use current data to revise and update the peer groups it uses to set Medi-Cal rates. In doing so, it should take into consideration the consolidation of the nursing facility industry.

10. To ensure that the MHSA-funded triage grants are effective, the Oversight Commission should require that local mental health agencies uniformly report data on their uses of triage grants. It should also establish statewide metrics to evaluate the impact of triage grants by July 2018.

10. To ensure that it provides the public with nursing facility information that is accurate and comprehensible, Health Planning should update its regulations to do the following:

- Append additional schedules to the template for the annual cost report to enable nursing facilities to fully disclose related-party transactions.

- Provide a single location in the annual cost report template for nursing facilities to enter related-party transaction amounts next to the amounts they are claiming for Medi-Cal reimbursement.

- Create an additional schedule in the cost report template that depicts how a company is investing in quality-of-care improvements.

4. To ensure that students with disabilities have equal access to instructional materials, by June 2018, the Chancellor's Office should develop guidance for the community colleges on periodically monitoring the accessibility of instructional materials and on providing training to all instructors in making their materials accessible to students with disabilities.

7. To assist all community colleges in increasing transparency of their shared governance decision-making processes, by September 2018, the Chancellor's Office should issue guidance to the community colleges on establishing procedures to document the attendees, input received, and agreements reached during department meetings, including those to consider technology equipment requests.

56. To ensure the health and safety of students in a laboratory setting, beginning in the Fall 2018 semester, San Diego should perform reviews at least annually to ensure that all departments are using the student training acknowledgement forms and are complying with the retention requirement.

3. To improve CSU's financial transparency with students and other stakeholders, the Chancellor's Office, with the approval of the trustees, should revise CSU policy by October 2019 to require that it publish information about CSU's discretionary surplus. At a minimum, the Chancellor's Office should revise its reserve policy to establish and justify a minimum sufficient level of reserve for economic uncertainty and require the Chancellor's Office to provide additional oversight to ensure that CSU maintains that level. This oversight should include monitoring, approving, and notifying the trustees of any uses of the reserve for economic uncertainty.

9. The Chancellor's Office should require that, by October 2019, the campuses publish the names of the alternate transportation committee members, the committee meeting minutes, and the committee meeting schedule on their parking and transportation services websites.

5. Once it has developed the health and safety reporting template and campuses have used it to submit their reports, the Chancellor's Office should develop recommendations for improving campus health and safety and follow up on the campuses' implementation of any corrective actions related to these recommendations.

6. Once it has developed the health and safety reporting template and campuses have used it to submit their reports, the Chancellor's Office should incorporate the risks identified in its assessments into the University Auditor's audit plan to ensure that the University Auditor evaluates problem areas related to campus health and safety.

10. To ensure the health and safety of employees working with hazardous materials, the Chancellor's Office should prescribe the frequency for which the campuses provide refresher laboratory safety training to employees.

59. Based on campuses annual reports on the timeliness of their inspections of safeguards, engineering controls, and ventilation systems, the Chancellor's Office should follow up with campuses that report untimely inspections and should require that the campuses develop action plans to ensure that they complete inspections as often as state regulations require.

3. To improve the oversight of CSU's management personnel, the Chancellor's Office should work with campuses, bargaining unit representatives, the Public Employment Relations Board, and others as necessary to come to an agreement on the appropriate classification of coaches. The Chancellor's Office should take into account the concerns that San Diego State has raised about the labor market for these employees.

10. The Chancellor's Office should finish developing the Common Human Resources System and implement it as scheduled by December 2019.

5. To determine the amount of money that it can reallocate to campuses and to ensure that it publicly presents comprehensive and accurate budget information, by April 2018 the Office of the President should implement our recommended budget presentation shown in Figure 11 on page 40. Specifically, the Office of the President's budget presentation to the regents should include a comparison of its proposed budget to its actual expenditures for the previous year. It should also include all its expenditures and identify changes to the discretionary and restricted reserves. The Office of the President should combine both the disclosed and undisclosed budgets into one budget presentation.

22. To determine the amount of money that it can reallocate to campuses and to ensure that it publicly presents comprehensive and accurate budget information, by April 2019 the Office of the President should continue to present a comprehensive budget based on the presentation in Figure 11 to the regents, the Legislature, and the public.

23. To ensure that its staffing costs align with the needs of campuses and other stakeholders, by April 2019 the Office of the President should set targets for any needed reductions to salary amounts using the results from its public and private sector comparison and adjust its salaries accordingly.

32. To determine the amount of money that it can reallocate to campuses and to ensure that it publicly presents comprehensive and accurate budget information, by April 2020 the Office of the President should evaluate its budget process to ensure that it is efficient and has adequate safeguards that ensure that staff approve and justify all budget expenditures. If the Office of the President determines that its safeguards are sufficient, it should begin developing a multiyear budget plan.

34. To ensure that its staffing costs align with the needs of campuses and other stakeholders, by April 2020 the Office of the President should adjust its salary levels and ranges to meet its established targets.

36. To ensure that its staffing costs align with the needs of campuses and other stakeholders, by April 2020 the Office of the President should reallocate funds to campuses when adjustments to its salaries and benefits result in savings.

39. To ensure that its staffing costs align with the needs of campuses and other stakeholders, by April 2020 the Office of the President should report to the regents on the amount of funds it reallocates to campuses as a result of implementing our recommendations.

8. To ensure that the university achieves its goals of obtaining services at the lowest cost or best value and of providing vendors with fair access to contracting opportunities, the Office of the President should revise the university's contract manual to incorporate the best practices found in the State Contracting Manual for limiting the use of amendments to repeatedly extend existing contracts.

9. To ensure that the university achieves its goals of obtaining services at the lowest cost or best value and of providing vendors with fair access to contracting opportunities, the Office of the President should revise the university's contract manual to narrow the exemption from competition to only selected professional services, similar to the State Contracting Manual.

15. To maximize benefits from the systemwide procurement initiative and to ensure that the university uses those benefits for its teaching, research, and public service missions, the Office of the President should study ways to measure actual procurement benefits—possibly focusing this effort on benefits from larger dollar amounts—and if such measurement is not possible, it should clearly disclose to the regents and the public that the amounts it reports are based on estimates.

16. To maximize benefits from the systemwide procurement initiative and to ensure that the university uses those benefits for its teaching, research, and public service missions, the Office of the President should, if actual benefits are measurable, implement a process to monitor and report annually to the regents the estimated and actual benefits.

24. To address any patterns or systemic problems of sexual harassment, by July 2019 the systemwide office should work with each campus to develop and implement processes and data reports to assist the campus in regularly identifying patterns and systemic problems related to sexual harassment and in instituting sexual harassment prevention education and training in those areas that need it.

25. To address any patterns or systemic problems of sexual harassment, by July 2019 the systemwide office should work with each campus to implement ongoing data quality control processes in order to ensure sexual harassment complaints data are accurate and complete.

26. To address any patterns or systemic problems of sexual harassment, by July 2019 the systemwide office should identify and review campuses' complaints data to identify outliers in their use of the formal, informal, and administratively closed processes.

1. To meet its commitment to California residents, the university should replace its "compare favorably" policy with a new admission standard for nonresident applicants that reflects the intent of the Master Plan. The admission standard should require campuses to admit only nonresidents with admissions credentials that place them in the upper half of the residents it admits.

2. To meet its commitment to California residents, the university should amend its referral process by taking steps to increase the likelihood that referred residents ultimately enroll.

3. To ensure that campuses' interpretations of admission standards do not adversely impact residents, the university should implement a thorough process to annually evaluate the qualifications of students who apply and students who are admitted. These evaluations should highlight instances when campuses admit nonresidents who are less qualified than residents and should include corrective action steps. Moreover, this evaluation should include resident and nonresident undergraduate enrollment in majors at each campus. The university should make the results of this evaluation—including details of the academic qualifications of students who applied and who were admitted—publicly available.

8. To ensure the reasonableness of the compensation the university pays its executives, it should include—to the extent possible— all items of compensation when setting or adjusting salaries and benefits, when conducting surveys and studies, and when comparing the compensation packages of its executives to those in similar positions outside the university.

10. To improve the transparency and timeliness of its annual compensation report, the university should streamline the process it uses to prepare the report so it can be issued by April of each year.

12. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should immediately require that the campuses fully participate in all projects unless they can provide compelling evidence demonstrating a harmful effect.

13. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should, by June 30, 2016, to the extent possible, implement a process to centrally direct these funds to ensure that campuses use them to support the core academic and research missions of the university.

14. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should ensure that it substantiates that projects are actually generating savings and new revenue and that it can demonstrate how the university uses these funds.

15. To ensure that its recruiting efforts benefit residents, the university should prioritize recruiting residents over nonresidents. In particular, the university should focus its recruiting efforts broadly to ensure that it effectively recruits resident underrepresented minorities. For example, the university could establish a limit on the amount of funds it dedicates to nonresident recruiting. Further, it should develop a process to better track its nonresident and resident recruiting expenditures.

16. To determine if the campuses are using funds to further the goals of the University of California system and the Legislature, the Office of the President should begin regularly monitoring and analyzing how campuses are using both state funds and nonresident supplemental tuition. If, after the close of the fiscal year, the Office of the President determines that campuses are not using state funds and/or nonresident supplemental tuition in accordance with those goals, the Office of the President should take steps to correct the campuses' spending decisions as soon as possible.

17. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should track spending from state funds for programs that do not relate to educating students.

18. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should reevaluate these programs each year to determine whether they continue to be necessary to fulfill the university's mission.

19. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should explore whether the programs could be supported with alternate revenue sources.

22. To ensure that its rebenching efforts lead to equalized per-student funding among the campuses, the university should adopt a methodology that it can use, at least every three to five years, to update its weighting system to ensure the weight factors take into account campuses' actual costs of instruction, using the cost study that we recommend in Chapter 1 and other revenue sources if necessary.

23. To ensure that its rebenching efforts lead to equalized per-student funding among the campuses, the university should exclude from its rebenching calculation all state funding it uses for programs that do not directly relate to educating students. The university should exclude these programs only after it has evaluated them in accordance with the recommendation we made previously.

7. To ensure the ongoing accountability of the Office of the President, the regents should require it to implement our recommendations and report periodically on its progress.

14. To ensure that the Office of the President's staffing levels are justified and that costs are reasonable and align with the needs of campuses and other stakeholders, the regents should require the Office of the President to implement our recommendations and report periodically on its progress.

4. UC Davis should collect all late fees that its licensees owe.

1. To strengthen its administrative reviews and help ensure that school food authorities comply with the Buy American requirement, Education should update its written procedures to include a requirement that reviewers collect and retain evidence for all items they evaluate for compliance with the Buy American requirement. This update should occur no later than October 1, 2017.

4. To make its appeal process more accessible to families who may not receive a satisfactory resolution from its contractors, Education should, by October 2018, require that its contractors share key information in their communications with families about the process for appealing notices. The required information should include valid grounds for a family to file an appeal as well as information or documentation Education would need in order to review the family's appeal of adverse decisions regarding their child-care services. Education should also require contractors to incorporate this information into contractually mandated staff training and into publicly available policies and procedures.

11. To ensure that its contractors can effectively make program improvements and maintain successes in ways that are meaningful to their stakeholders, Education should adopt measures to ensure its contractors follow the terms of their contracts by demonstrating that their board members conduct a critical appraisal of each education program.

23. To better understand the condition of school libraries statewide and to raise stakeholders' awareness of the State Education Board's adopted model standards, Education should identify school districts that reported employing significantly fewer teacher librarians in fiscal year 2015-16 than in previous years and verify the accuracy of their fiscal year 2015-16 reports.

8. To ensure that districts, county offices, and schools receive guidance on a variety of safety issues and to comply with state law, CDE and DOJ should resume their partnership activities, as required by state law. Further, the partnership should update the 2002 handbook, "Safe Schools: A Planning Guide for Action," and distribute it to all districts and county offices. If CDE or DOJ determine the need for additional funds to implement the legislative recommendations or to reestablish the partnership's activities, they should request those funds from the Legislature.

3. To ensure that all LEAs comply with federal special education requirements, Education should require them to include directly in a student's IEP document reasons for any changes to student placement or services.

8. To enable it to review additional areas of its special education program for quality assurance, Education should collect information about the frequency of the provision of each service contained in all students' IEPs. Education should then use this information to annually review the frequency of mental health services and follow up with SELPAs when it observes a significant reduction in the frequency of services.

9. To ensure that LEAs comply with federal and state requirements, Education should require all LEAs to use the IEP document to communicate the rationale for residential treatment and any potential harmful effects of such placement.

18. Education should analyze and report to the Legislature, by May 30, 2016, on the outcomes for students receiving mental health services statewide, including outcomes across the six performance indicators we identified, in order to demonstrate whether those services are effective. Once it has reported this statewide information, Education should provide each LEA throughout the State a report regarding the outcomes for the students the LEA served.

20. To ensure that the State knows the amount LEAs spend to provide mental health services for student IEPs, before the start of the 2017-18 fiscal year, Education should develop, and require all LEAs to follow, an accounting methodology to track and report expenditures related to special education mental health services.

29. To ensure that the State provides special education and related services to all eligible students, Education should investigate the difference between the estimated number of school aged children statewide who have a severe emotional disturbance and the number receiving mental health services through an IEP and determine the reason for such a discrepancy. Education should then take any steps necessary to assist LEAs in identifying and providing services to children who are severely emotionally disturbed.

3. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should distribute complaints and appeals to the correct divisions for investigation or review.

4. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should establish a single database to record and track all investigations of complaints and reviews of appeals. This database should capture all data necessary for Education to effectively make informed decisions related to UCP complaints or appeals. At a minimum, the database should capture the date on which Education received each complaint or appeal, the date on which it forwarded the complaint or appeal to the appropriate division for investigation or review, and the date on which it sent the decision to the complainant. The database should also include the type of complaint or appeal, the LEA involved, and the decision.

5. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should track the divisions' progress in processing complaints and appeals to ensure the divisions meet all UCP requirements, including documenting exceptional circumstances that constitute good cause for extending investigations beyond 60 days.

6. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should work with divisions to establish policies and procedures for the divisions to follow when investigating UCP complaints and reviewing appeals. The procedures should identify the individuals or units responsible for investigating complaints and reviewing appeals, the steps and time frames for conducting investigations and reviews, the requirements for issuing decisions, and the documentation that should be retained in the files.

7. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should establish and distribute a standard investigation report format that includes the required elements for the divisions to use when processing UCP complaints.

8. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should monitor the divisions' decisions and reports on complaints and appeals to ensure that they comply with requirements.

16. To increase the efficiency and effectiveness of LEAs' UCP processes, Education should work with those LEAs throughout the State that receive a disproportionately high number of non-UCP complaints through the UCP process to assess the potential benefits of establishing similar mechanisms.

22. To ensure that its regulations are consistent and align with state and federal requirements, Education should revise its regulations to allow LEAs to extend investigations under exceptional circumstances that constitute good cause if the LEAs document and support with evidence the reasons for the extensions.

23. After it makes the recommended regulatory changes to allow extensions under exceptional circumstances, Education should review LEAs' extensions to investigations as part of its Federal Program Monitoring to ensure that LEAs' documentation is sufficient and that their reasons adequately justify such extensions.

2. To reduce the risk of identity theft for its claimants before it completes its modernization project, EDD should, by December 2021, implement one or more of our proposed solutions or another viable solution to discontinue its use of full SSNs as unique identifiers on all documents that it mails to claimants. Further, it should prioritize addressing documents with the highest mail volumes, and it should make changes to these documents by March 2020. When providing us with the status of its implementation of this recommendation at 60 days, six months, and one year after the issuance of this report, and annually thereafter, EDD should note which documents it has addressed since the release of our report, how it has addressed them, and the dates by which it expects to address the remaining documents containing full SSNs that it mails to claimants.

3. To ensure that it eliminates any unnecessary uses of personal information in its external communications and to ensure that it fully protects its claimants' privacy, EDD should, by May 2019, implement its recently developed plan for reviewing new, revised, and existing documents. EDD should provide documents to us indicating the progress it has made to implement this recommendation at 60 days, six months, and one year following the release of this report. Finally, it should, by December 2021, complete its full review of existing documents and remove any unnecessary instances of personal information.

7. To minimize the degree to which its process to change its regulations may result in the disparate treatment of card room owners, the bureau should temporarily approve or deny its backlogged games applications by July 2019.

12. To better align the revenue in the Gambling Fund with the costs of the activities that the fund supports, the bureau and the commission should conduct cost analyses of those activities by July 2020. At a minimum, these cost analyses should include the following:

-The entities' personnel costs, operating costs, and any program overhead costs.

-Updated time estimates for their core and support activities, such as background investigations.

-The cost of their enforcement activities.

Using this information, the bureau and commission should reset their regulatory fees to reflect their actual costs. Before conducting its fee study, the bureau should implement our recommendations to improve its processes for assigning applications, ensuring the completeness of applications, and developing time-reporting protocols.

21. To ensure that it can provide useful and accurate data on the locations where enforcement employees spend their time, the bureau should equip its time-reporting system by November 2019 with the capacity to track all hours employees spend at each card room and casino

19. To ensure that it has complete disposition information, Justice should coordinate with the Judicial Council at least once a year to share information about court reporting gaps and to determine the need to distribute additional information to courts about reporting requirements and the manner in which to report. In addition, Justice should reconvene its advisory committee and meet on a regular basis to discuss, at a minimum, improving the frequency and timeliness with which courts report dispositions to Justice and law enforcement agencies report arrest information to Justice.

20. To ensure that it is receiving all arrest information from law enforcement agencies, at a minimum, Justice should consider trends in the number of arrest reports each law enforcement agency sends it and the number of reports that it might expect to receive from an agency given the agency's size, location, and reporting history. Whenever Justice identifies a law enforcement agency that it determines may not be reporting all required information, it should request that the agency forward all required arrest information.

3. To increase the effectiveness of hate crime prevention and response efforts, DOJ should provide additional guidance to law enforcement agencies by adding region-specific data fields to the hate crime database, including items such as the zip code in which reported hate crimes took place and other fields that DOJ determines will support its outreach efforts.

4. To increase the effectiveness of hate crime prevention and response efforts, DOJ should provide additional guidance to law enforcement agencies by analyzing reported hate crimes in various regions in the State and sending advisory notices when it detects hate crimes happening across multiple jurisdictions. It should also seek the resources to implement these efforts, if necessary.

5. To increase the effectiveness of hate crime prevention and response efforts, DOJ should provide additional guidance to law enforcement agencies by creating and disseminating outreach materials so law enforcement agencies can better engage with their communities.

6. To increase the effectiveness of hate crime prevention and response efforts, DOJ should provide additional guidance to law enforcement agencies by creating and making available training materials for law enforcement agencies on how best to identify and respond to hate crimes.

7. To ensure that law enforcement agencies effectively engage with communities regarding hate crimes, DOJ should provide guidance and best practices for law enforcement agencies to follow when conducting hate crime outreach to vulnerable communities within their jurisdictions, such as collaborating with a county human rights commission. It should make the outreach materials available to law enforcement agencies and should include in them presentation materials for various types of communities, including immigrants and Muslims, among others. It should seek the resources to implement these efforts, if necessary.

9. To ensure that districts, county offices, and schools receive guidance on a variety of safety issues and to comply with state law, CDE and DOJ should resume their partnership activities, as required by state law. Further, the partnership should update the 2002 handbook, "Safe Schools: A Planning Guide for Action," and distribute it to all districts and county offices. If CDE or DOJ determine the need for additional funds to implement the legislative recommendations or to reestablish the partnership's activities, they should request those funds from the Legislature.

10. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, developing best practices based on the requirements stated in the federal regulations, the state guidelines and state law, and advising user agencies on the implementation of those practices. The best practices should include, but not be limited to reviewing criminal intelligence, appropriately disseminating information, performing robust audit practices, establishing plans to recover from disasters, and meeting all of the State's juvenile notification law requirements. Justice should guide the board and the committee to develop these best practices by June 30, 2017.

11. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing user agencies that use CalGang to complete a comprehensive review of all the gangs documented in CalGang to determine if they meet the necessary requirements for inclusion and to purge from CalGang any groups that do not meet the requirements. Justice should guide the board and the committee to ensure that user agencies complete this review in phases, with the final phase to be completed by June 30, 2018.

12. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing all user agencies to complete a comprehensive review of the records in CalGang to determine if the user agencies have adequate support for the criteria associated with all the individuals they have entered as gang members. If the user agencies do not have adequate support, they should immediately purge the criteria—and, if necessary, the individuals—from CalGang. In addition, the user agencies should ensure that all the fields in each CalGang record are accurate. Justice should guide the board and the committee to ensure that user agencies complete this review in phases, with the final phase to be completed by September 30, 2019.

13. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing all user agencies to report to Justice every six months, beginning in January 2017, on their progress toward completing their gang and gang member reviews.

2. Unless the Legislature directs the board to eliminate the compliance fund's excess fund balance within a time frame of more than a year, the board should eliminate the excess fund balance by June 30, 2017 by using it to offset the licensing program's annual funding shortfall. The board should also limit the fund's future balance to no more than two months' worth of licensing program expenditures.

6. To help address California's e-commerce tax gap and further ensure out-of-state retailers' compliance with state law regarding nexus, Tax Administration should implement a two-year pilot of its authorized reward program for information resulting in the identification of unreported sales and use taxes.

1. The authority should amend its regulations to bring them into accord with the 2004 act, thus allowing any eligible hospital to apply for the 2004 act's funds that remained as of June 30, 2014.

6. To ensure that it adequately investigates alleged judicial misconduct, by April 2020 CJP should expand the role of its legal advisor's office to include periodic reviews of the quality of closed investigations and, as warranted, to recommend changes to CJP's investigative practices.

8. To allow it to detect potential judicial misconduct associated with legal errors, CJP should immediately direct its staff to use more appropriate allegation codes when closing complaints at intake. By October 2019, CJP should determine what data it will need to begin tracking so it can trend information—voluntarily provided by complainants—that could indicate complaints about legal error should be investigated because there is a risk that legal error is the result of underlying misconduct, such as bias. By October 2019, CJP should also develop procedures that indicate how often it will evaluate its data for such trends and establish guidelines for when trends warrant CJP staff recommending that the commission open an investigation. CJP should begin tracking that information and implement these procedures as soon as possible.

10. To improve its transparency and accessibility to the general public, by April 2020 CJP should implement a plan to regularly engage in outreach activities that target the general public.

12. To ensure that it expeditiously improves the public's ability to submit complaints, CJP should begin accepting complaints online upon updating its electronic case management system.

14. To maximize the resources available for its core functions, CJP should immediately begin exploring options for relocating its office to a less expensive location and relocate as soon as possible.

15. To ensure that it obtains the resources necessary to fulfill its mission, CJP should report to the Legislature by May of each of the next three years about the following:

- Its progress in implementing our recommendations and any associated effects on its workload.

- The steps it has taken to realize efficiencies in its operations.

- Its evaluation of whether the investigations manager is a full-time position and any funding it will need in the future to support that position.

- Its progress in purchasing and implementing a new electronic case management system.

- Its progress in relocating its office space to a more affordable location.

- Any savings or unforeseen costs arising from the changes we identify above.

1. To ensure that the compensation the AOC provides is reasonable, the Judicial Council should adopt procedures that require a regular and thorough review of the AOC's compensation practices including an analysis of the job duties of each position to ensure that the compensation aligns with the requirements of the position. This review should include comparable executive branch salaries, along with a justification when an AOC position is compensated at a higher level than a comparable executive branch position.

3. To ensure that its compensation structure is reasonable, the AOC should mirror the executive branch's practices for offering leave buyback programs in terms of frequency and amount.

4. To increase its efficiency and decrease its travel expenses, the AOC should require its directors and managers to work in the same locations as the majority of their staff unless business needs clearly require the staff to work in different locations than their managers.

11. To reduce its expenses, the AOC should cease its excessive reimbursements for meals by adopting the executive branch's meal and travel reimbursement policies.

14. To ensure that it spends funds appropriately, the AOC should develop and implement controls to govern how its staff can spend judicial branch funds. These controls should include specific definitions of local assistance and support expenditures, written fiscal policies and procedures as the rules of court require, and a review process.

19. The AOC should conduct a comprehensive survey of the courts on a regular schedule—at least every five years—to ensure that the services it provides align with their responses. The AOC should re-evaluate any services that the courts identify as being of limited value or need.

20. To justify its budget and staffing levels, the AOC should conduct the steps in CalHR's workforce planning model in the appropriate order. It should begin by establishing its mission and creating a strategic plan based on the needs of the courts. It should then determine the services it should provide to achieve the goals of that plan. The AOC should base its future staffing changes on the foundation CalHR's workforce planning model provides. Finally, the AOC should develop and use performance measures to evaluate the effectiveness of this effort.

21. To ensure that it provides services to the trial courts as efficiently as possible, the Judicial Council should explore implementing a fee-for-service model for selected services. These services could include those that are little used or of lesser value to the trial courts, as identified in our survey that we discuss in Chapter 3.

22. To justify the budget and staff level of the AOC, the Judicial Council should implement some or all of the best practices we identified to improve the transparency of AOC spending activities.

11. The Secretary of State should adopt regulations establishing clear criteria for mistakes in election-related materials that constitute reportable errors and require counties to report these errors to it after each election.

12. Beginning in December 2018, the Secretary of State should implement annual risk-based reviews of a selection of county election officials' offices to ensure their compliance with state election laws and regulations.

13. To inform and enhance the guidance it provides to county election officials, the Secretary of State should analyze error reports and its risk-based review results to focus its guidance on topics most relevant to improving elections throughout the State.

9. To enable it to effectively determine its budget, State Bar should continue to annually prepare five-year projections.

10. To ensure that it maximizes the revenue from its San Francisco building, State Bar should lease all available space and ensure that its leases reflect market rates.

11. To ensure that it maximizes the revenue from its San Francisco building, in the event of any future staff growth, State Bar should avoid adding space by reducing its space allocations when practical to more closely match industry standards.

14. To further its ability to operate more efficiently and reduce the backlog of discipline cases, State Bar should use its performance measures and collected data going forward to evaluate its case processing goals and work with the Legislature to revise the 180-day statutory goal if necessary.

6. To assign purchasing cards only to appropriate staff, ensure that the State Bar's records of employees' credit limits reflect those established with the bank, and to verify that staff use purchasing cards only for allowable and necessary expenses, the State Bar should immediately develop a policy that requires justification of the business needs for employees to receive purchasing cards, and use this policy to limit the number of staff issued a purchasing card.

7. To assign purchasing cards only to appropriate staff, ensure that the State Bar's records of employees' credit limits reflect those established with the bank, and to verify that staff use purchasing cards only for allowable and necessary expenses, the State Bar should immediately restrict the use of purchasing cards to its original purpose, which was for low-dollar and frequently occurring purchases. For purchases above $5,000, the State Bar should require the vendor to bill for payment.

9. To ensure that its costs are reasonable and appropriate, the State Bar should update its meal and catering policy to align with the meal policy of the State's Executive Branch and should require individuals attending committee meetings for the State Bar to comply with standard meal per diem rates.

22. By April 2019, the SCO should review attendance for the employee for the time period that she worked an alternate schedule to determine whether she owes the State any additional hours as a result of informally switching her regular day off (RDO). If she does, the SCO should recover any overpayments or adjust her leave balances accordingly.

23. By June 2019, the SCO should review the attendance records of the manager's other staff to determine whether the other employees also failed to account for any missed work time.

24. The SCO should determine whether other managers or supervisors at the SCO also allowed employees to informally switch their RDOs. If so, the SCO should review the attendance records for the relevant employees to verify that they accurately recorded their time off and hours worked. To the extent that the SCO determines other employees improperly accounted for their time, the SCO should recover any overpayments or adjust their leave balances accordingly.

2. The Los Angeles court should ensure that it documents best value in its procurement files when selecting vendors from leveraged procurement agreements.

13. To ensure that it properly authorizes payments and purchases only allowable items, the San Mateo court should process payments in accordance with the requirements and recommended practices of the Judicial Council and the State. Specifically, the San Mateo court should amend its bottled water service contract to ensure that water is purchased for use by jurors and court room staff only.

7. To ensure that DWR manages WaterFix in an effective manner, DWR should complete both the economic analysis and financial analysis for WaterFix and make the analyses publicly available as soon as possible.

4. To enable policymakers and the public to track the Authority's progress toward meeting the federal grant deadline of December 2022, the Authority should, by January 2019, begin providing quarterly updates to the Legislature detailing the progress of the three Central Valley construction projects using an earned value model that compares construction progress to the projected total completion cost and date. The Authority should base these updates on the most current estimates available.

5. To ensure that it is adequately prepared if it is unable to meet the federal grant deadline of December 2022, the Authority should, by May 2019, develop a contingency plan for responding to such a scenario.