Appendix

SCOPE AND METHODOLOGY

The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to examine DHCS’ oversight of the rural expansion and of managed care in the Regional Model counties. Specifically, the Audit Committee directed us to identify the process DHCS used to create the Regional Model, determine whether the level of care health plans have provided the Regional Model’s beneficiaries has been acceptable, and identify factors that may prevent the Regional Model counties from establishing a COHS. The table below lists the objectives that the Audit Committee approved and the methods we used to address them.

Audit Objectives and the Methods Used to Address Them
AUDIT OBJECTIVE		METHOD
1	Review and evaluate the laws, rules, and regulations significant to the audit objectives.	Reviewed relevant federal and state laws, rules, and regulations related to DHCS’ oversight of managed care, health plans’ acceptable delivery of managed care, and the establishment of a COHS.
2	Identify the process by which DHCS identified and grouped the 18 counties in question into the Regional Model and evaluate the reasonableness of the process.	Interviewed DHCS staff to identify the process it used to transition the 28 rural expansion counties, including the 18 Regional Model counties, to managed care. Interviewed representatives of the rural expansion counties, including the Regional Model counties, to determine how their counties learned they would be transitioning from fee-for-service to managed care, what types of interactions they had with DHCS, and whether DHCS addressed any concerns or health plan preferences they had. Evaluated any efforts DHCS made to communicate with counties regarding the managed care transition process. Reviewed and evaluated the process DHCS used to group the 18 counties into the Regional Model and whether that process was reasonable.
3	For the past three years, assess the rates of claims being paid by the Regional Model commercial plans and how they compare to Medi-Cal managed care plans offered through the COHS Model.	Evaluated available fiscal years 2015–16 through 2017–18 financial records for Anthem, Health & Wellness, and Partnership to determine the amounts they spent to provide services to their beneficiaries. Interviewed DHCS staff to determine how it sets capitation rates. Evaluated the differences between the benefit packages for the Regional Model and the COHS Model and the effect that the benefit packages had on the amounts DHCS paid those models’ health plans per beneficiary. Evaluated Medi-Cal cost data from fiscal years 2013–14 through 2016–17 for all 18 Regional Model counties and eight Partnership counties to determine how much DHCS spent to deliver services to the beneficiaries of those counties.
4	Determine how DHCS selected the Regional Model commercial plans, review the terms of any relevant agreements, and assess the degree to which DHCS considered stakeholder input or other relevant factors.	Interviewed DHCS staff to determine the process it used to select the Regional Model health plans. Evaluated whether DHCS followed the applicable laws when it selected the Regional Model health plans. We determined that DHCS followed relevant laws when it selected Anthem and Health & Wellness to provide services in the Regional Model counties. Evaluated DHCS’ method for requesting feedback from stakeholders before it selected the health plans, as well as the extent to which DHCS addressed that feedback during its selection process.
5	For the counties served under the Regional Model, determine the following:	Analyzed DHCS’ statewide alternative access standard data to determine whether Anthem and Health & Wellness provided beneficiaries in the Regional Model with access to care that was comparable to other parts of the State. Analyzed statewide HEDIS data from 2015, the earliest year data was available, through 2018 to determine how the quality of care Anthem and Health & Wellness provided beneficiaries in the Regional Model changed since its implementation and whether that care was comparable to other parts of the State. Reviewed DHCS’ and Managed Health Care’s audit reports to determine whether the care that Anthem and Health & Wellness provided was similar to the care provided by other plans operating in rural expansion counties. Analyzed DHCS’ provider directory data to calculate the number of providers with which Anthem, Health & Wellness, and Partnership contracted.
	a. Whether the level of care in those counties is disproportionately low as compared to other parts of California. To the extent possible, determine whether and how the level of care has changed since the implementation of the Regional Model.
	b. Whether the level of care received is acceptable as it relates to industry standards and state and federal requirements.	Interviewed staff at DHCS and Managed Health Care to identify criteria defining an acceptable level of care. Reviewed DHCS’ and Managed Health Care’s audit reports of Anthem and Health & Wellness to determine whether the health plans met state, federal, and contractual requirements. Analyzed HEDIS data from 2015 through 2018 to determine whether Anthem and Health & Wellness met the minimum performance levels that DHCS required. Analyzed DHCS’ alternative access standard data to determine whether Anthem and Health & Wellness provided beneficiaries in the Regional Model with access to care that met state requirements. We were unable to identify the number of beneficiaries whose access to care exceeded the state requirements because DHCS could not provide us with records that identified the number of beneficiaries assigned to each health plan by zip code.
	c. Whether DHCS has taken steps to ensure that the plans adhere to the provisions of their contracts and whether DHCS has provided that information to the counties.	Reviewed DHCS’ policies and procedures related to medical audits and corrective action plans. Determined the extent to which DHCS made its monitoring results available to counties and potential stakeholders. Evaluated DHCS’ efforts to notify counties and potential stakeholders of its monitoring and of the results of that monitoring. Interviewed a selection of Regional Model and Partnership county representatives to obtain their perspectives on DHCS’ efforts to notify them of its monitoring results.
	d. Whether opportunities exist to improve the current level of care Medi-Cal beneficiaries receive under the Regional Model.	Interviewed DHCS staff to determine whether DHCS has identified opportunities to improve the Regional Model’s level of care. Evaluated DHCS’ policies and procedures related to alternative access standards and network certification CAPs to identify opportunities to reduce access barriers. Evaluated the extent of DHCS’ authority to require health plans to take corrective actions. Compared provider data from the Medical Board of California and the Osteopathic Medical Board of California to DHCS’ provider directory data to determine whether Anthem and Health & Wellness have contracted with all of the available providers located in the Regional Model counties. Evaluated the characteristics of DHCS’ managed care models to determine whether any were better suited than others to serve the Regional Model counties.
6	Determine whether DHCS, when negotiating and extending its contract with the Regional Model commercial plans, made efforts to consider and mitigate any concerns communicated to DHCS by affected counties. Assess whether the process was sufficiently transparent.	Interviewed DHCS staff and a selection of Regional Model county staff to determine whether DHCS requested feedback from the counties before it extended Anthem’s and Health & Wellness’s contracts.
7	Evaluate what compels the Regional Model counties to remain in the existing commercial plan model as opposed to creating or joining a COHS.	Evaluated DHCS’ contracts with Anthem and Health & Wellness to determine whether they require the counties to remain in the Regional Model. Interviewed DHCS staff and other personnel at selected Regional Model and Partnership counties, Partnership, and Gold Coast to identify the processes for joining or establishing a COHS, the cost of establishing a COHS, and the entities responsible for funding the establishment of a COHS. Evaluated federal and state laws to determine whether they impose any limitations on DHCS’ contracting with an additional COHS.
8	Review and assess any other issues that are significant to the audit.	Interviewed DHCS staff to determine its process for approving or denying alternative access standards. Evaluated DHCS’ policies and procedures for reviewing alternative access standard requests. Evaluated a selection of 30 alternative access standard requests to determine whether DHCS adhered to its policies and procedures when it approved them.

Source: Analysis of the Audit Committee’s audit request number 2018-122, state law, and information and documentation identified in the column titled Method.

Assessment of Data Reliability

The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of the computer‑processed information that we use to support our findings, conclusions, and recommendations. In performing this audit, we relied on DHCS’ provider directory, alternative access standard data, and HEDIS performance data to evaluate the access to care and quality of care that the Medi‑Cal managed care health plans provided to their beneficiaries. Additionally, we relied on license and eligibility data from the Medical Board of California and the Osteopathic Medical Board of California in order to identify licensed medical providers who are eligible to contract with Medi‑Cal. To evaluate these data, we performed electronic testing of the data, reviewed existing information about the data, interviewed agency officials knowledgeable about the data, and performed data set verification procedures. We found that the DHCS provider directory, alternative access standards, and HEDIS performance data were sufficiently reliable for the purposes of our audit.

However, during our review, we identified limitations with the Medical Board of California and Osteopathic Medical Board of California license data. Specifically, we found that the license data limited the number of practice locations for each provider and that not all providers submitted this information. As a result, we found the license data were of undetermined reliability for identifying the practice location of all providers. Although this determination may affect the precision of some of the numbers we present, there is sufficient evidence in total to support our findings, conclusions, and recommendations.