Appendix A

Technical Appendix: Data and Methodology

Our analysis of the effect of Corrections' CBT rehabilitation programs on recidivism rates for inmates released in fiscal year 2015–16 drew on data from Corrections and DOJ. During fiscal year 2014–15, Corrections offered all four CBT programs at 11 men's prisons. We compared recidivism rates and other data for inmates that received CBT classes to the data for those that did not. Our analysis examined inmates who completed CBT programs but excluded inmates who completed programs for long‑term offenders, defined as inmates with an indeterminate sentence with the possibility of parole. We used a regression analysis to control for many observable characteristics between the test and control groups in an effort to isolate the effect on the recidivism rates of inmates completing at least half of their assigned CBT classes. Due to limitations in Corrections' data, we were able to examine only the one‑ and two‑year recidivism rates. In developing our methodology we consulted with the director of the Center for Evidence‑Based Corrections at UC Irvine, and a statistician reviewed our statistical model and its results.

Data

Our analysis used data from both SOMS and DOJ. Corrections implemented its SOMS inmate database statewide in October 2014. It was designed to consolidate existing databases and replaced multiple manual paper processes to standardize adult inmate data and inmate population management practices. SOMS includes personal information on individual inmates, such as their date of birth, education level, and current location within the prison system. SOMS also includes information on the programming inmates received, such as when an inmate is enrolled in or completed a rehabilitation class. We also obtained conviction data from DOJ to determine whether an inmate recidivated within two years of his release from prison.

Methodology

We obtained SOMS data to calculate recidivism rates for inmates released in fiscal year 2015–16. Corrections began using SOMS to track whether inmates completed CBT classes in October 2014. Due to this data limitation, we were able to select only inmates that completed CBT classes from October 2014 through their release in fiscal year 2015–16 to determine whether the inmates recidivated within two years.¹² Thus, inmates released in fiscal year 2015–16 are the earliest release group our analysis could examine and still calculate the one‑ and two‑year recidivism rates.

Our analysis required inmates to have completed CBT classes for at least half of their needs as the minimum before being considered to have had their rehabilitative needs met. Corrections assessed inmate needs in four CBT categories (anger management, substance abuse, criminal thinking, and family relationships). Inmates who take classes that address at least half of their assessed needs should have reduced their likelihood to recidivate because Corrections' contracts require vendors to use only evidence‑based programs, meaning that the classes have been shown to be effective. We used half, rather than all needs met, due to the limited number of prisons that offered CBT classes during the time period we examined. If we had focused only on inmates who had all of their needs met, it would have severely limited the sample size for our analysis.

To create the control and test groups for our analysis, we split the cohort from the 11 prisons that were released in fiscal year 2015–16 into inmates who had at least half of their CBT needs met and inmates who had not been assigned to CBT classes. Corrections' policies prioritize inmates' assignment to any CBT classes based on a combination of their COMPAS and CSRA scores. Our analysis examined inmates who were rated as having a moderate to high need in at least one of the CBT categories. Given that our test and control groups represented subsets of the prison population, we took three steps to adjust for the potential differences in characteristics between the two groups. First, we grouped inmates within the test group by the total number of their CBT needs and their level of need for each of the four CBT categories. Second, we matched a selection of inmates from the control group with each member of the test group based on their CBT needs combination. We did this to ensure that we compared inmates who had the same CBT needs and who differed in whether or not they completed at least half of the classes they were assigned. Finally, we controlled for important observable differences between the control group and the test group, including age, self‑identified race, education (as measured by the inmate's most recent TABE reading score and whether or not the inmate had a verified high school diploma or equivalency), crime risk (as measured by the inmate's CSRA score), and prison assignment with variables in our regression analyses, as discussed below.

We calculated the overall recidivism rates between the control and test groups with a regression analysis of the correlation between inmates completing at least half of their assigned CBT classes (treatment) and recidivism rates using probit regressions. Our regressions examined the entire sample of the control and test groups to determine the relationship between completing CBT classes and inmates' one‑ and two‑year recidivism rates while controlling for prison location, age, self‑identified race, education, and crime risk. We examined the correlation between completing CBT classes and the one‑ and two‑year recidivism rates for each subset of our control variables. This means that we conducted individual regressions for prison, self‑identified race, education level, age, and crime risk. Table A shows the results of these regressions, including the percentage change in recidivism for inmates completing CBT classes and the probability (p-value) that the results could be caused by chance. The likelihood of obtaining a result that appears to be significant, but actually occurs by chance, increases with the total number of regressions performed. Nonetheless, we are confident in our analysis that did not find an overall relationship between inmates completing CBT rehabilitation programs and their recidivism rates.

Table A
P‑Values for Our Analysis of Inmates Who Completed CBT Classes for at Least Half of Their Needs
Factor	Category	P‑value*	Percentage decrease or increase in One‑year recidivism	P‑value	Percentage decrease or increase in Two‑year recidivism
*Overall*		0.285		0.228

*Prison*	Avenal State Prison	0.806		0.671
	California City Correctional Facility	0.288		0.821
	California Institution for Men	0.894		0.656
	California Men's Colony	0.636		0.136
	Correctional Training Facility	0.411		0.789
	Chuckawalla Valley State Prison	0.936		0.162
	High Desert State Prison	0.471		0.677
	Ironwood State Prison	0.750		0.162
	California State Prison, Los Angeles County	0.086^†	‑15%	0.063^†	‑18%
	California Substance Abuse Treatment Facility	0.016^‡	‑10%	0.000^§	‑18%
	Valley State Prison	0.319		0.732

*Self‑Identified Race*	Black	0.049^‡	‑5%	0.027^‡	‑7%
	Hispanic	0.672		0.776
	White	0.379		0.974
	Other	0.554		0.583

*CSRA*	Low Risk	0.145		0.915
	Moderate Risk	0.969		0.857
	High Risk ‑ Drug	0.086^†	11%	0.199
	High Risk ‑ Property	0.106		0.794
	High Risk ‑ Violence	0.002^§	‑13%	0.001^§	‑16%

*Education*	No education	0.129		0.357
	Elementary School	0.530		0.992
	Middle School	0.314		0.245
	Freshman/Sophomore	0.051^†	‑6%	0.006^§	‑11%
	Junior/Senior	0.807		0.456
	High school diploma/GED	0.278		0.083^†	‑6%
	No High school diploma/GED	0.452		0.592

*Age*	18 to 29 years old	0.255		0.269
	30 to 39 years old	0.090^†	‑5%	0.324
	40 to 49 years old	0.458		0.604
	50+ years old	0.707		0.958

Source: Regression output based on SOMS and DOJ data.

* The p‑values test the null hypothesis that inmates' assignment to at least half of their needed CBT classes has no effect. The lower the p-value, the more likely you are to reject the null hypothesis. A higher p‑value indicates that you cannot reject the hypothesis. In other words, a low p‑value is likely to be meaningful. Therefore, we only present recidivism increases or decreases for p‑values of 0.10 or less.

^† p < 0.1

^‡ p < 0.05

^§ p < 0.01

Appendix B

Scope and Methodology

The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to perform an audit to examine the effectiveness of in‑prison rehabilitation programs at Corrections. Table B outlines the Audit Committee's objectives and our methods for addressing them.

Assessment of Data Reliability

The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of computer‑processed information that we use to support our findings, conclusions, and recommendations. In performing this audit, we obtained Corrections' inmate information to identify inmates' demographics, rehabilitative needs, locations, and program participation. Additionally, we obtained DOJ's statewide conviction information to identify inmates who recidivated. To evaluate these data, we performed electronic testing of the data, reviewed existing information about the data and systems, and interviewed agency officials knowledgeable about the data. However, during our review we identified data limitations. Specifically, program participation information was not centrally tracked until the current system was implemented in October 2014. Additionally, we were unable to identify the needs of some inmates because Corrections does not perform assessments on all inmates. Therefore, we found these data to be of undetermined reliability for the audit purposes. Although this determination may affect the precision of the numbers we present, there is sufficient evidence in total to support our findings, conclusions, and recommendations.

Table B
Audit Objectives and the Methods Used to Address Them
AUDIT OBJECTIVE		METHOD
1	Review and evaluate the laws, rules, and regulations significant to the audit objectives.	Identified and reviewed the laws, rules, and regulations for the Inspector General, C‑ROB, Corrections, and CalPIA that were applicable to Corrections' in‑prison rehabilitation programs.
2	Identify the roles and responsibilities of Corrections and any other parties related to the oversight of state‑funded rehabilitation programs and assess the adequacy of their oversight, including the extent to which they ensure appropriate and consistent implementation across institutions.	Documented the Inspector General and C‑ROB's efforts in reviewing Corrections' rehabilitation programs. In addition, we identified the statutorily mandated oversight responsibility of the Inspector General and C‑ROB and found they were generally complying with those requirements. Reviewed a selection of 10 academic education programs and determined they were accredited with the Western Association of Schools and Colleges. Evaluated the adequacy of oversight of prison rehabilitation programs by Corrections, Inspector General, C‑ROB, CalPIA, and the Prison Industry Board.
3	Determine whether Corrections has conducted an assessment to determine the level of resources required to meet the rehabilitative needs of inmates in all of Corrections' facilities.	Obtained and evaluated assessments conducted by Corrections from 2012 through 2018 to determine if Corrections is meeting its resource allocation goals and whether those goals are adequate to meet the rehabilitative needs of its inmates.
4	Review and evaluate the effectiveness of the rehabilitation programs by determining the following:
	a. Corrections' justification for the program's use and whether the program is based on evidence and research.	Reviewed and documented the process used by Corrections when procuring CBT services. Reviewed 10 CBT vendor contracts to determine if Corrections procured those services appropriately. Selected 10 contracts to determine whether the vendors were using evidence‑based curricula. In addition, we tested three vendors' contracts to determine whether the curricula being taught at the prisons were the same as the curricula maintained in Corrections' database.
	b. Corrections' method for evaluating the cost‑effectiveness of the program and whether it has considered investing in an independent oversight entity to perform this function.	Interviewed Corrections staff and reviewed policies and procedures to evaluate the methodology for assessments Corrections and CalPIA have conducted regarding the cost‑effectiveness of their rehabilitation programs. Reviewed and evaluated best practices from other state and federal entities to determine an appropriate methodology for Corrections and CalPIA to follow for determining the cost‑effectiveness of its rehabilitation programs. Interviewed Corrections and CalPIA staff to determine if they have considered using an independent oversight entity to evaluate the cost‑effectiveness of their rehabilitation programs. Evaluated the objectives and methodology of the proposed study that UC Irvine is conducting on the impact of Corrections' rehabilitation programs on recidivism.
5	Determine whether Corrections maintains adequate data on its rehabilitation programs—such as enrollment, attendance, and outcomes, by facility—to allow stakeholders to compare the effectiveness of rehabilitation resources across facilities.	Obtained a complete list of the rehabilitation programs offered at all prisons, and interviewed staff to determine how long the agency has offered the programs, and how Corrections measures program completeness (Corrections operates 36 prisons at 35 locations; the Folsom location houses two prisons, a men's prison—Folsom State Prison—and a women's prison—Folsom Women's Facility). Obtained SOMS data for all inmates incarcerated from October 2014 through September 2018 and determined the number of inmates enrolled, attending, and completing rehabilitation programs. Obtained all rehabilitation program vendor contracts for fiscal year 2017–18 and determined the budgeted capacity for all programs offered by Corrections. Compared the number of enrolled inmates to the budgeted capacity for each program during fiscal year 2017–18 at the three prisons we reviewed. Determined that Corrections maintains adequate data on rehabilitation programs, including enrollment, attendance, and outcomes, by facility to allow stakeholders to compare effectiveness of rehabilitation resources across facilities. Analyzed CalPIA data from July 2014 through October 2018 to determine its vocational programs' capacity, enrollment, and vacancy rate.
6	Determine whether Corrections effectively assesses inmates' risks and rehabilitation needs and reviews and evaluates how it ensures that the tools used to identify the needs and risks are valid. Assess the adequacy of Corrections' policies, procedures, and practices for selecting and prioritizing inmates for participation in rehabilitation programs.	Interviewed Corrections staff and reviewed policies, procedures, and practices regarding the assessment and prioritization of inmates' needs. Reviewed and evaluated assessments (COMPAS, CSRA, and TABE) used by Corrections and determine whether the tests have been validated. Selected 60 inmates incarcerated by Corrections from fiscal years 2015–16 through 2016–17 to determine whether Corrections followed established policies and procedures to place inmates into appropriate rehabilitation programs. Examined how Corrections prioritized inmates for rehabilitation programs for 19 inmates during fiscal year 2017–18 at the three prisons we selected to determine if the inmate assignments into their respective rehabilitation programs were appropriate.
7	Identify and assess Corrections' performance measures—such as whether inmates' rehabilitative needs were met prior to their release and whether inmates are progressing in their programs—to evaluate the effectiveness of rehabilitative services.	Obtained all policies and reports regarding performance measures, including historical documents. Determined the recidivism rate for inmates that Corrections released from prison in fiscal year 2015–16 and compared the rate of recidivism for those inmates that had at least half of their rehabilitation needs met to those that had none of their needs met. Identified best practices for performance measures used by other states, federal agencies, and countries applicable to California. Determined what issues are preventing Corrections from working with EDD to track inmates' employment outcomes post‑release. Reviewed whether Corrections' policies are tracking academic educational performance through academic testing. Although Corrections does track inmates' academic performance, it does not set performance measures or determine whether inmates' rehabilitative needs were met prior to release.
8	Determine whether Corrections maintains a waiting list for rehabilitative programs, the number of inmates on the waiting list, how long they have been on the waiting list, their risks, and their needs.	Interviewed correctional counselors at the three prisons we selected and determined how staff place inmates on waiting lists at each institution. Determined if Corrections' waiting list process effectively prioritizes inmates with a moderate to high risk of recidivating and a moderate to high need into rehabilitation programs. Determined the number of inmates on waiting lists for academic, vocational, and CBT programs, how long they have been on the waiting list, and their risks and needs.
9	Review and assess any other issues that are related to the audit.	Interviewed Corrections staff and obtained a complete list of all volunteer programs and determined whether Corrections evaluated these programs for their effectiveness.

Source: Analysis of Audit Committee's audit request number 2018‑113, planning documents, and analysis of information and documentation identified in the table column titled Method.

APPENDIX C

The Number of Inmates on Waiting Lists and the Average Time Spent on the List

We reviewed Corrections' waiting list data, and we present the number of inmates on waiting lists as of July 2018 in Table C. This table shows that there were between 18,000 and 31,000 inmates on waiting lists as of July 2018, and that those inmates were on waiting lists for an average of between 211 and 351 days. However, these data are presented for informational purposes only because, as we discuss, we identified significant flaws with Corrections' waiting lists.

Table C
The Number of Inmates on Waiting Lists and the Average Time Spent on the List
Waiting list that addresses the need for:*	Number of inmates on waiting list	Average number of days on waiting list
*Academic education*
Target population^†	7,178	230
Not target population	10,820	338
Totals	17,998	295
*Vocational education*
Target population^†	6,702	285
Not target population	23,008	351
Totals	29,710	336
*Anger management*
Target population^†	10,951	217
Not target population	19,983	268
Totals	30,934	250
*Criminal thinking*
Target population^†	9,554	215
Not target population	21,524	261
Totals	31,078	247
*Substance abuse disorder treatment*
Target population^†	12,796	211
Not target population	18,172	289
Totals	30,968	257
*Family relationships*
Target population^†	15,733	212
Not target population	13,313	295
Totals	29,046	250

Source: Analysis of SOMS data as of July 24, 2018.

* We excluded programs designated for inmates serving lengthy periods of incarceration, such as life with the possibility of parole.

^† Corrections defines its target population as inmates with a moderate to high risk of recidivating and a moderate to high need for the program.

Footnote

¹² Individual CBT classes generally last three to four months and no longer than six months. Inmates would have had the opportunity to take multiple classes before their release during fiscal year 2015–16. Go back to text