Report 2012-122 Recommendation 11 Responses

Report 2012-122: Mental Health Services Act: The State's Oversight Has Provided Little Assurance of the Act's Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance (Release Date: August 2013)

Recommendation #11 To: Mental Health Services Oversight and Accountability Commission

To fulfill its charge to evaluate MHSA programs, the Accountability Commission should undertake the evaluations specified in its implementation plan.

Annual Follow-Up Agency Response From November 2017

The five-year Evaluation Master Plan adopted by the Commission in 2013 served well as an initial strategic guide for research and evaluation. The Implementation Plan that accompanied the Evaluation Master Plan was designed to be a living document to allow for adding and revising the list of evaluations over the timeline of the plan in response to environmental events. The Commission in 2015 contracted with University of California, Davis and University of California, Los Angeles to conduct the last two studies listed on the Implementation Plan for FY 2017/18, the fifth year of the five-year plan. The UC Davis contract to look at early psychosis was completed in June of 2017. The last deliverable (fact sheets) on the UC Los Angeles contract looking at the effectiveness of programs for older adults is due December 30, 2017.

The initial phase is now completed. The Commission is working to develop metrics for monitoring outcomes, tracking reductions in disparities and developing tools to support broader public accountability, each of which are referenced in the MHSA. The MHSA outlines specific goals including reductions in unemployment, homelessness, school failure, out of home placements, suicide, criminal justice involvement, and prolonged suffering. The Commission is working on developing strategies to measure progress against those goals. Such work includes conducting policy research projects similar to the recently completed project on the intersection of the criminal justice and mental health systems that evaluated the challenges and opportunities to reducing the number of adults with mental health needs in the criminal justice system and improving outcomes for those who are in custody. These projects that integrate evaluation and data analyses will be identified and prioritized by the Commission through an open and deliberative process.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From September 2015

The Evaluation Master Plan, adopted by the Commission on March 2013, is a five year plan and will not be fully implemented until fiscal year 2017-2018. Since August 2013 when the BSA issued this recommendation, the Commission has undertaken the evaluations specified in the Commission's implementation plan for this period of time (i.e. fiscal years 2013-2014 and 2014-2015). At this point in time, two months into the 2015-16 fiscal year, the evaluations for the current fiscal year have not yet begun. However, the Commission followed its normal process and at the October 23, 2014 meeting voted on specific evaluation activities to take place in fiscal year 2015-2016.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2014

The Evaluation Master Plan, adopted by the Commission on March 2013, is a five year plan and will not be fully implemented until fiscal year 2017-2018. Since August 2013 when the BSA issued this recommendation, the Commission has undertaken all of the evaluations specified in the Commission's implementation plan for this period of time (i.e. fiscal years 2013-2014 and 2014-2015). At its October 23, 2014 meeting the Commission voted on the specific evaluation activities to take place in fiscal year 2015-2016.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

The Evaluation Master Plan, adopted by the Commission on March 2013, is a five year plan and will not be fully implemented until fiscal year 2017-2018. Since August 2013 when the BSA issued this recommendation, the Commission has undertaken all of the evaluations specified in the Commission's implementation plan for this period of time (i.e. fiscal years 2013-2014 and 2014-2015) and has begun to plan for evaluation activities to take place in fiscal year 2015-2016.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

The Evaluation Master Plan, adopted by the Commission on March 2013, is a five year plan and will not be fully implemented until fiscal year 2017-2018. Since August 2013 when the BSA issued this recommendation, the Commission has undertaken all of the evaluations specified in the Commission's implementation paln for this period of time (i.e., fiscal year 2013-2014) and has begun to plan for evaluation activities to take place in fiscal year 2014-2015.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

The Evaluation Master Plan, adopted by the Commission in March 2013, is a five year plan and will not be fully implemented until fiscal year 2017-2018. Since the BSA issued this recommendation, the Commission has undertaken the evaluations specified in the implementation plan for this period of time. The Commission has:

a. Continued progress on the UCLA contract for priority indicators

b. Continued progress on the UCD contract for reducing disparities in access to care

c. Continued progress on the UCLA contract for PEI

d. Continued progress on the RDA contract for the community program planning

e. Entered into a contract with CiMH for INN evaluations

f. Entered into a contract with MHDA for CSI data collection and reporting strengthening and support

g. Issued a contract with MHDA and DHCS for IT strengthening of the DCR, which is owned by DHCS

h. Began refining the measurements of existing indicators

i. Began negotiating a contract to determine effectiveness of methods for engaging and serving TAY clients

j. Issued an RFP to develop a system to track outcomes for persons in less intensive services than FSP

California State Auditor's Assessment of 60-Day Status: Partially Implemented

All Recommendations in 2012-122

Agency responses received are posted verbatim.