Report 2020-102 Recommendation 12 Responses

Report 2020-102: Public Safety Realignment: Weak State and County Oversight Does Not Ensure That Funds Are Spent Effectively (Release Date: March 2021)

Recommendation #12 To: Alameda County

To ensure that the programs and services funded by public safety realignment funds are effective, beginning immediately, Alameda should conduct evaluations of the effectiveness of its programs and services at least every three years.

Annual Follow-Up Agency Response From March 2025

Alameda County is committed to evaluating public safety realignment programs every three years, starting in 2025. Building on insights from a 2023 Request for Information (RFI), the County will issue a Request for Proposal (RFP) this year to select an external research firm for a comprehensive triennial evaluation of AB 109-funded programs. The RFP will prioritize researchers with lived experience who reflect the County's diversity.

The evaluation will occur in two phases. Phase one will assess data access, quality, and availability across programs, updating the County's inventory of AB 109-funded services to capture theories of change, service details, spending, and performance measures. It will also evaluate agency readiness and explore data standardization. Findings will inform recommendations to improve data collection, retention, and access.

Phase two will use a Results-Based Accountability approach to assess program implementation, fidelity, effectiveness, return on investment, and impact on recidivism. It will examine factors influencing client outcomes, quality of life improvements, and intersectional cultural considerations such as race, ethnicity, gender identity, and sexual orientation.

This evaluation builds on past assessments, including the Results First initiative and the Risk-Need-Responsivity Simulation Tool Assessment, which have shaped key programmatic changes, such as expanded mental health and substance use interventions and the Elevate Academy. Prior findings show AB 109-funded services reduce recidivism in Alameda County.

Beyond its existing quality assurance measures, the County's triennial evaluation will enhance accountability, ensure effective use of funds, and strengthen evidence-based practices to improve client outcomes and community safety.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

As we have noted in our previous assessments, the State Auditor's recommendation is not limited to evaluations of only those programs funded through the Community Corrections account, as Alameda described in its previous responses. Rather, to fully implement this recommendation, Alameda should evaluate the programs it funds through all of its public safety realignment accounts. Alameda's current response is unclear as to whether it intends to evaluate all public safety realignment funds as part of its triennial evaluation. Further, Alameda did not provide documentation of the triennial evaluation for us to assess whether it sufficiently addresses our recommendation. Until Alameda provides documentation that demonstrates its review of all programs and services funded by public safety for effectiveness, we will continue to report this recommendation as not fully implemented. We look forward to reviewing Alameda's progress toward implementing this recommendation during its next annual response.

1-Year Agency Response

On November 15, 2021, the Community Corrections Partnership Executive Committee approved a plan to evaluate the County's AB109 programs on a 3-year rotation. The plan identified a schedule to ensure all programs will be evaluated, prioritized by time in service and set on a rotation schedule to ensure that all programs and services are evaluated every 3 years. The first step of this plan is to develop an up-to-date inventory of all programs and services that require evaluation, which will cover AB 109-funded programs from the Community Corrections subaccount, for which it has jurisdiction. ACPD is currently working with County justice partners to update this inventory and it is expected to be complete by 4/30/2022. Once the program inventory is updated, an RFP will be developed to secure an evaluator. Additionally, Alameda County Probation is working with George Mason University (GMU) to conduct a Risk-Need-Responsivity (RNR) Simulation Tool Assessment of AB109 Providers. GMU used its RNR Simulation Tool to assess the extent the county's AB109 providers' service models are evidence-based and how they are meeting the population's needs. The RNR Simulation Tool assesses the level and dosage of services, adoption of EBPs, and implementation issues, identifying areas where services can be strengthened. AB 109 programs have completed this assessment and received program reports which detail strengths, improvement areas, and recommendations across six domains related to best practices. This program information, in combination with data from ACPD about the AB109 population (e.g., risk level, criminogenic needs, other responsivity factors or service need areas), is currently under analysis to understand the county's capacity to meet the needs of the population through these programs. These analyses identify critical gaps or areas where program availability may be mismatched with population needs. Reporting of these results is expected to occur by Summer 2022.

California State Auditor's Assessment of 1-Year Status: Pending

The State Auditor's recommendation is not limited to evaluations of only those programs funded through the Community Corrections account, as Alameda describes in its response. Rather, to fully implement this recommendation, Alameda should evaluate the programs it funds through all of its public safety realignment accounts. We look forward to reviewing Alameda's progress toward implementing this recommendation during its next annual response.

6-Month Agency Response

The Community Corrections Partnership Executive Committee will develop an inventory of all programs and services that require evaluation, prioritized based on length of time in service; and provide a schedule that ensures all identified programs and services are evaluated every 3 years. This will cover AB 109-funded programs from the Community Corrections subaccount, for which it has jurisdiction. The CCPEC will establish this schedule by 12/31/2021. Additionally, Alameda County Probation is working with George Mason University (GMU) to conduct a Risk-Need-Responsivity (RNR) Simulation Tool Assessment of AB109 Providers. GMU will use its RNR Simulation Tool to assess the extent these AB109 providers' service models are evidence-based and how they are meeting the population's needs. The RNR Simulation Tool has a program assessment which can assess the level and dosage of services, adoption of EBPs, and implementation issues, identifying areas where services can be strengthened. After completing this assessment, the program receives a report that: (1) classifies the program based on information provided; (2) identifies the type of clients that are most suitable for the program; (3) rates the program in six areas related to program quality and fidelity to EBPs and (4) provides recommendations to improve the program quality within each of these domains. This program information will then be used in combination with data from ACPD about the AB109 population (e.g., risk level, criminogenic needs, other responsivity factors or service need areas) to assess the county's capacity to meet the needs of the population through these programs. These analyses identify critical gaps or areas where program availability may be mismatched with population needs. This process is anticipated to take 30 months to complete.

California State Auditor's Assessment of 6-Month Status: Pending

The State Auditor's recommendation is not limited to evaluations of only those programs funded through the Community Corrections account, as Alameda describes in its response. Rather, to fully implement this recommendation, Alameda should evaluate the programs it funds through all of its public safety realignment accounts. We look forward to reviewing Alameda's progress toward implementing this recommendation during its 1-year response.

60-Day Agency Response

The Community Corrections Partnership Executive Committee will develop an inventory of all programs and services that require evaluation, prioritized based on length of time in service; and provide a schedule that ensures all identified programs and services are evaluated every 3 years. This will cover AB 109-funded programs from the Community Corrections subaccount, for which it has jurisdiction. The CCPEC will establish this schedule by 12/31/2021. Additionally, Alameda County Probation is working with George Mason University (GMU) to conduct a Risk-Need-Responsivity (RNR) Simulation Tool Assessment of AB109 Providers. GMU will use its RNR Simulation Tool to assess the extent these AB109 providers' service models are evidence-based and how they are meeting the population's needs. The RNR Simulation Tool has a program assessment which can assess the level and dosage of services, adoption of EBPs, and implementation issues, identifying areas where services can be strengthened. After completing this assessment, the program receives a report that: (1) classifies the program based on information provided; (2) identifies the type of clients that are most suitable for the program; (3) rates the program in six areas related to program quality and fidelity to EBPs and (4) provides recommendations to improve the program quality within each of these domains. This program information will then be used in combination with data from ACPD about the AB109 population (e.g., risk level, criminogenic needs, other responsivity factors or service need areas) to assess the county's capacity to meet the needs of the population through these programs. These analyses identify critical gaps or areas where program availability may be mismatched with population needs. This process is anticipated to take 30 months to complete.

California State Auditor's Assessment of 60-Day Status: Pending

The State Auditor's recommendation is not limited to evaluations of only those programs funded through the Community Corrections account, as Alameda describes in its response. Rather, to fully implement this recommendation, Alameda should evaluate the programs it funds through all of its public safety realignment accounts. We look forward to reviewing Alameda's progress toward implementing this recommendation during its 6-month response.

All Recommendations in 2020-102

Agency responses received are posted verbatim.