Report 2017-129 Recommendation 16 Responses

Report 2017-129: Department of Rehabilitation: Its Inadequate Guidance and Oversight of the Grant Process Led to Inconsistencies and Perceived Bias in Its Evaluations and Awards of Some Grants (Release Date: July 2018)

Recommendation #16 To: Rehabilitation, Department of

To ensure that Rehabilitation has appropriate oversight of its grant process and can sufficiently demonstrate that it followed the process, it should designate staff, separate from those involved in the respective grant process, to conduct a review of each grant process for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications before it awards grants.

Annual Follow-Up Agency Response From September 2023

The Department is currently developing a Request for Applications for Traumatic Brain Injury and expects to award grants in or about July 2024. Thereafter, the Department will submit documentation demonstrating full implementation of this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2022

The Department finalized a RFA for the Traumatic Brain Injury program in March 2022 and provided guidance and tools for use during the post evaluation review to the post evaluation review staff member. The Department will provide the CSA contact with documentation of the correspondence sent by the program director to the post evaluation review staff member providing guidance and tools to ensure understanding of the process. This correspondence will demonstrate full implementation of the CSA recommendation after grant award.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Although Rehabilitation provides some guidance to its post evaluation reviewer, it does not describe how the department expects the reviewer to conduct their review. For example, it provides the reviewer with the evaluators' qualifications and self-certifications, indicating that they do not have a conflict of interest. However, the guidance does not describe its expectations of the steps the reviewer should take to ensure that these certifications are accurate. Because we identified potential conflicts of interest in this audit and another subsequent grant through basic internet searches, we expected that Rehabilitation would provide comprehensive guidance to its reviewer with the necessary steps to verify that evaluators are free from prejudice. Thus, until it does so, we will report this recommendation as not fully implemented.

We look forward to reviewing Rehabilitation's progress toward implementing this recommendation during future responses.

Annual Follow-Up Agency Response From October 2021

Please see the Department's response to CSA recommendations #3, #10 and #13 for specific improvements to our process which are reflected in our current Request for Application (RFA) for the Traumatic Brain Injury Program September 2021 process. Further, the Department will provide more guidance to the post evaluation staff person assigned to review the RFA process thoroughly. Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2020

The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Although Rehabilitation designated staff separate from those involved in the grant process to conduct a review of the process for its most recent grant, the evaluation notes that the team did not detect procedural errors or evaluator bias or prejudice. However, based on our review of available documentation, we found several discrepancies in the grant process, such as the RFA not including a tiebreaker methodology, staff involved in the grant process who may not have completed ethics training, and an evaluator with at least a perceived bias. As a result, we do not believe that Rehabilitation's post-evaluation review was comprehensive enough to demonstrate that Rehabilitation followed its grant process or to ensure sufficient oversight of the process.

Annual Follow-Up Agency Response From October 2019

The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2017-129

Agency responses received are posted verbatim.