Appendix

Scope and Methodology

The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to examine the oversight and regulation of the QME process by DWC. The table below lists the objectives that the Audit Committee approved and the methods we used to address them.

Audit Objectives and the Methods Used to Address Them
AUDIT OBJECTIVE		METHOD
1	Review and evaluate the laws, rules, and regulations significant to the audit objectives.	Researched and reviewed relevant laws, rules, regulations, and policies.
2	Identify the number of QMEs and the number of requests for their services during the last five fiscal years. To the extent possible, determine whether the current number of QMEs can meet the demand for the program’s services. Determine what efforts, if any, DWC has made to increase the number of QMEs participating in the program.	Analyzed data from the department’s database to identify the number of QMEs and requests for their services for fiscal years 2013–14 through 2017–18. Reviewed trends of replacement panels that DWC generated during fiscal years 2013–14 through 2017–18 caused by QME unavailability. Interviewed key staff regarding DWC’s efforts to increase the number of participating QMEs. Interviewed an applicants’ attorneys organization to gain an understanding of injured workers’ access to QME services.
3	Review DWC’s policies and procedures for appointing and reappointing QMEs and determine whether they comply with state laws and regulations. Review a selection of QME appointments and reappointments to determine whether DWC is following its processes and complying with statutory requirements.	Reviewed relevant laws and regulations for appointing and reappointing QMEs. Reviewed a selection of 20 QME appointments and reappointments for 2018 and 2019 to determine whether DWC complied with state laws and regulations when it appointed and reappointed QMEs. Reviewed a selection of 20 QME reappointment denials (five from the selection in the bullet point above and 15 additional denials) for 2018 and 2019 to determine the reasons DWC denied reappointments. Interviewed key stakeholders within the workers’ compensation system, including an applicants’ attorneys association, a defense attorneys’ firm, and DWC’s chief judge to gain an understanding on one of the issues we identified related to reappointment: poor‑quality medical‑legal reports by QMEs.
4	Review DWC’s policies and procedures for QMEs to appeal denial of appointments and reappointments and determine whether they comply with state laws and regulations. Review a selection of appeals by QMEs to determine whether DWC is following its processes and complying with statutory requirements.	Reviewed relevant laws and regulations that allow applicants and QMEs to appeal denial of appointments and reappointments, including their rights to hearings. Reviewed a selection of 10 reappointment denial appeals QMEs submitted for 2018 and 2019 to determine if DWC complied with regulations when it processed the appeals. Reviewed an additional selection of 10 reappointment denial appeals in which DWC provided to QMEs a statement of issues to uphold the denial from 2016 through 2018 to determine whether DWC complied with its regulations to grant hearings when requested.
5	Assess the process used by DWC to establish its existing fee schedules for the program and determine whether the process complies with state laws and regulations. In addition, evaluate the process, if any, DWC uses to update or modify program fee schedules. To the extent possible, assess the reasonableness and fairness of the fee schedules.	Reviewed state laws for updating DWC’s Official Medical Fee Schedule and medical‑legal fee schedule. Evaluated the process DWC is currently using to revise its medical-legal fee schedule and determined when DWC last updated the rates. We considered the increases in inflation when assessing the reasonableness of DWC’s rates from 2006—the last time DWC updated its rates.
6	Review DWC’s process for investigating and resolving complaints filed by QMEs when insurance carriers deny or reduce their payments for services. Review a selection of complaints filed by QMEs to determine whether DWC followed its process and complied with statutory requirements.	Reviewed state laws and regulations related to the independent bill review process and DWC’s independent bill review contract. Reviewed a selection of 12 independent bill review cases, which are initiated to resolve a QME’s billing-related dispute or complaint, and found that from 2015 through 2018, DWC’s independent bill review contractor generally followed its process and complied with statutory requirements.
7	Assess how DWC allocates its audit resources and prioritizes reviews of QMEs and insurance carriers.	Compared the roles, responsibilities, and staffing for DWC’s audit unit—which audits insurance carriers—and investigations section—which reviews QMEs. Based on the separate organizational structure, staff, and roles and responsibilities, we concluded that the audit unit and the investigations section are separate. The evidence did not indicate that DWC prioritized its investigations section or its audit unit over the other.
8	Evaluate the relationship between DWC, the insurance industry, and the independent bill review process. To the extent that specific concerns are identified, review communications between DWC and the various parties.	Reviewed DWC’s contract with its independent bill review contractor from 2015 through 2019 to gain an understanding of the contracted services and DWC’s involvement in the bill review process. Reviewed communications between relevant DWC staff, the independent bill review contractor, and insurance carriers. Our review did not identify any inappropriate communications.
9	Determine whether DWC collected funds on behalf of the insurance industry from QMEs and distributed those funds to insurance carriers or other parties. To the extent it occurred, determine whether DWC’s collection and payment of those funds complied with state laws and regulations.	Selected and reviewed 19 billing complaints that DWC received from 2014 through 2018. For each of the billing-related complaints we reviewed, we determined whether correspondence and settlement documents directed QMEs to pay restitution to insurance carriers and determined whether restitution amounts were recorded in DWC’s financial records. Reviewed DWC’s financial records from 2016 through 2018 to identify the source of DWC’s collections. We found that DWC generally directed QMEs to make restitution payments to insurance carriers and we did not identify instances where DWC deposited the money in its account.
10	Review and assess any other issues that are significant to the audit.	Reviewed 24 billing-related complaints (19 selected from Objective 9 and five additional items) to determine whether DWC followed its disciplinary and reappointment processes.

Source: Analysis of Audit Committee’s audit request number 2019‑102, state law, and information and documentation identified in the column titled Method.

Assessment of Data Reliability

The Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of the computer‑processed information that we use to support our findings, conclusions, and recommendations. In performing this audit, we relied on the department’s QME database to determine the number of available QMEs and requests for QMEs. To evaluate these data, we reviewed existing information about the data, interviewed agency officials knowledgeable about the data, and performed data set verification procedures and electronic testing of key data elements. In addition, we performed accuracy and completeness testing related to available QMEs. As a result, we found the data to be sufficiently reliable for our purposes.