Report 2005-111 All Recommendation Responses

Report 2005-111: California Department of Corrections and Rehabilitation: The Intermediate Sanction Programs Lacked Performance Benchmarks and Were Plagued With Implementation Problems (Release Date: November 2005)

Recommendation #1 To: Corrections and Rehabilitation, Department of

When planning future intermediate sanction programs, the parole division should decide on appropriate benchmarks for monitoring performance, identify the data it will need to measure performance against those benchmarks, and ensure that reliable data collection mechanisms are in place before a program is implemented. After implementing a new intermediate sanction program, the parole division should analyze the data it has collected and, if relevant, use the data in existing databases to monitor and evaluate the program's effectiveness on an ongoing basis.

Annual Follow-Up Agency Response From November 2012

November 28, 2012

In August 2011, the following community-based programs were transferred from the California Department of Corrections and Rehabilitation (CDCR), Division of Adult Parole Operations (DAPO), to the CDCR, Division of Rehabilitative Programs (DRP): Parolee Service Centers, Residential Multi-Service Centers, Day Reporting Centers, Community-Based Coalition, Substance Abuse Treatment and Recovery, and Computer Literacy Learning Labs.

As mentioned in previous responses to the Bureau of State Audits, DRP uses a

“Key Performance Indicator” (KPI) system to measure performance outcomes, and was in the process of incorporating these programs into its KPI system in order to measure performance one year ago. It is important to note that these programs joined those already under the jurisdiction of DRP, including the Substance Abuse Services Coordination Agencies, the Female Offender Treatment and Education Program, and the Parolee Services Network.

Recommendation No. 1

In October 2012, DRP established appropriate performance indicators/benchmarks for the community-based programs it inherited from DAPO. These include, but are not limited to, the following:

• Enrollment

• Attendance

• Program Completion

All community-based programs are required to submit these data to DRP on a monthly basis, via a secure upload to a database provided by CDCR. DRP's Performance, Data and Fidelity Unit then publishes the data in its monthly KPI reports. DRP's executive and management team analyzes the KPI reports to assess utilization. Where issues or concerns are identified, DRP shares that information with DAPO, in an effort to increase enrollment, attendance and completion.

However, at this time, DRP publishes only the “Enrollment” data collected from the programs transferred from DAPO. The Division is in the process of analyzing how best to group the “Attendance” and “Program Completion” data, since there is a fair amount of variation in the way that the current contractors track these data elements.

In its prior response, DRP indicated that “Recidivism Rate/Reduction” would also be established as a performance indicator. This element has not yet been fully implemented, for a number of reasons. Though CDCR's Office of Research publishes annual reports examining the recidivism rates of individuals released from prison, CDCR does not yet have the resources to track recidivism by individual program.

DRP and CDCR's Office of Research continue to explore options to do this, so that we can differentiate outcomes by the various interventions. Unfortunately, given current staffing and priority workload issues, this is not something that CDCR can commit to accomplishing in the near future.

Other potential performance indicators could include employment, education levels, and characteristics of those served. DRP anticipates KPIs will be finalized by July 2013.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Recommendation #2 To: Corrections and Rehabilitation, Department of

The parole division should ensure that the savings estimates developed during program planning are based on reasonable assumptions. If those assumptions change, it should update the savings estimates promptly.

1-Year Agency Response

The parole division concurs with our recommendation and indicates it will ensure that any discussions with legislative staff or other researchers include reasonable projections or estimates, and that it updates and reassesses projected savings in a timely manner. Specifically, when developing its fiscal year 2006-07 budget, the parole division indicates adjusting the assumptions and savings estimates related to its parole programs based on current data. (2007-406, p.241)

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Recommendation #3 To: Corrections and Rehabilitation, Department of

The parole division should consider analyzing the effect programs have had on parolee behavior and should use the knowledge it gains from the analyses to make future intermediate sanction programs more effective. The analysis should include the benefits of adding features to make these programs more effective.

Annual Follow-Up Agency Response From January 2008

The Division of Adult Parole Operations (DAPO) currently has a contract with the San Diego State University Research Foundation (SDSURF) to evaluate four programs, which ends 12-31-08:

- Day Reporting Centers/Community Cased Coalitions

- In-Custody Drug Treatment Program

- Parolee Service Centers

- Restitution Centers

As mentioned in the response to Recommendation 1, DAPO currently has two in-house project analysts paid for by contract dollars that assist in development of data collection requirements, collect and analyze data on our programs. They work with new contractors to develop databases for data collection and are setting up a data warehouse for DAPO. DAPO has a database that creates a consolidated data file for each program. They are currently able to produce a variety of data reports on DAPO programs, which are used to monitor and evaluate program effectiveness on an ongoing basis. DAPO has used data to make changes in programs to make them more effective. Utilizing this data, DAPO will collaborate with the Office of Research to have a report done for 2006 on DAPO parolee programs.

Both the 2003, "An Evaluation of the California Preventing Parolee Crime Program" and the 2006 CDCR Response to the BSA Audit showed the longer parolees stayed in programs/or-completed programs the less likely they were to return to custody. DAPO is focusing on incentives and disincentives to encourage parolee retention in/completion of programs.

The Parole Service Center (PSC) program has made several modifications to program contracts as a result of the recommendations of the BSA and as a result of a DAPO workgroup that cited and produced several noteworthy recommendations. The PSC Program Data Collection Form (PDCF) allows staff to collect critical data required for, both, program monitoring and future evaluation purposes. The PDCF is a comprehensive form that collects static, dynamic and subjective information on each parolee who enters the program. This form has been approved by the San Diego State University (SDSU) evaluation team, led by Dr. Sheldon Zhang and is the key data source for the future evaluation of the PSC Program. Second, the PSC now uses the Register of Program Participation (RPP) form and is mandated for use by each PSC contractor. This form captures benchmark data, such as program referrals, enrollments, occupancy and house of program instruction each parolee receives on a monthly basis. This form complies with a previous BSA finding which suggested that contractor accountability was difficult to assess. Additional program modifications now include the use of Sign In/Sign Out sheets for all programming performed at or off grounds of a PSC. The program also implemented a 52-week, certified Batterer's Program for those parolees required to complete this program as a condition of parole.

The In Custody Drug Treatment Program (ICDTP) is a redesign of the Substance Abuse Treatment Control Units (SATCU) designed to allow for the use of evidence-based research to reduce recidivism. The components include Phase I, which provides a 60-day in-custody, educationally based drug treatment education phase (increased from previous 30 day program). Phase II is a 30-day residential aftercare treatment program component provided by community based organizations, and transitions into Phase III which is a 60-day community based nonresidential treatment phase. One of the issues raised in the BSA audit related to the SATCU program was low utilization level. Monthly meetings are held to review statistics, including bed capacity levels. In addition, DAPO staffs assigned to the ICDTP program maintain daily capacity records. Reports are sent out weekly and monthly so that participation can be constantly monitored.

An informal evaluation is being conducted by SDSU researchers on DAPO's employment programs to identify which program components are more effective so future contracts better address the employment needs of parolees and is expected to be completed by December of 2008. One employment program that consistently failed to meet the benchmarks is being redesigned to be more effective.

The 2007 Expert Panel on Adult Offender and Recidivism Reduction Programming Report to the California Legislature assessed five DAPO programs using the California Program Assessment Process (CPAP). Based on this evaluation, DAPO is reviewing the programs to make improvements. (2007-041, p. 51-52).

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

All Recommendations in 2005-111

Agency responses received after June 2013 are posted verbatim.