Appendix

Scope and Methodology

We conducted this audit pursuant to the audit requirements contained in the Business and Professions Code. For this audit, we assessed the State Bar’s management of its attorney discipline system by reviewing its staffing levels, the timeliness of its investigations, and its disposition of cases, as well as the transparency of the information it provides to stakeholders in its discipline report. We also reviewed the State Bar’s response to the COVID‑19 pandemic and its impact on the state bar examination and on prospective California attorneys. The following table lists the audit objectives and the methods we used to address them.

Audit Objectives and the Methods Used to Address Them

AUDIT OBJECTIVE		METHOD
1	Review and evaluate the laws, rules, and regulations significant to the State Bar’s operations.	Reviewed relevant laws, regulations, and other background materials.
2	Evaluate the State Bar’s management of its attorney discipline system, including but not limited to the following: The resources, including the level of staff devoted to the attorney discipline system. The timeliness of its investigations and disposition of cases. The effectiveness of this system in protecting the public from attorneys who engage in inappropriate conduct. The level of transparency the State Bar provides in its reports to the Legislature that provide various measures of its discipline process.	Reviewed the State Bar’s policies and interviewed staff regarding its processes for investigating and disciplining attorneys accused of misconduct, including its reorganization of its trial counsel’s office beginning in 2016, its implementation of a new case prioritization process in 2018, and its implementation of the new case management system in 2019. Assessed the total backlogged cases for each month of the audit period and interviewed staff to determine reasons for the increasing backlog and case processing times. Calculated the average number of case processing days by phase and case priority level, and the number of pending cases per month by phase to determine correlations between timeliness and staff workload. Determined the outcomes of closed cases and calculated the percentage of closed cases that involved disciplinary actions. Evaluated internal benchmarks and performance metrics for the trial counsel’s office and reviewed a selection of 10 backlog cases that the State bar processed during 2018 and 2019. We could not test the majority of the cases processed in 2018 against the State Bar’s internal benchmarks because its case management system did not contain the information necessary, such as when an initial interview was conducted. We tested five cases processed in 2019 and found that three did not meet at least one of the internal benchmarks. However, the State Bar could not provide specific reasons for these delays, and stated that it had suspended some internal benchmarks to alleviate increased staff workload due to an increase in complaints and changes made in anticipation of its new case management system. Reviewed and analyzed the State Bar’s discipline reports and interviewed staff to determine if the reports included accurate, consistent, and sufficient data regarding the attorney discipline system.
3	Summarize and evaluate the State Bar’s recent efforts to manage its revenue reserves and expenditures to fully support its public protection mission.	Reviewed the State Bar’s general fund revenues, expenditures, and reserve balance from 2015 through 2020 and documented changes that led to a 2020 general fund surplus. Compared a selection of the State Bar’s travel and reimbursement policies against California Department of Human Resource’s requirements and found no material differences. We also assessed the State Bar’s process for using its real property based on our previous audit recommendations.
4	Identify the impact of the COVID‑19 pandemic on the bar exam and on prospective California attorneys by determining the following: The State Bar’s response to the pandemic and its work with stakeholders to administer the exam safely, responsibly, and in a timely manner. Best practices of other state bars to administer their respective exams.	Reviewed Supreme Court letters and decisions involving changes to the 2020 bar exam and approval of a provisional licensure program as a result of the COVID‑19 pandemic. Reviewed and analyzed the State Bar’s policies and documents and interviewed staff to identify how the State Bar planned, implemented, and administered the October 2020 bar exam and the provisional licensure program. Reviewed and assessed New York’s, Oregon’s, and Texas’s responses to the COVID‑19 pandemic and compared their actions to the steps California took to modify its admission practices and its administration of the bar exam in response to the pandemic. We found California took steps similar to those taken by these states. Reviewed the 2018 and 2019 statements of economic interests filed by the board, the Committee of Bar Examiners, and the State Bar’s executive management and admissions division staff. We determined that none of these individuals disclosed a financial interest in ExamSoft.
5	Review and assess any other issues that are significant to the audit.	Reviewed and analyzed the State Bar’s procurement manual and interviewed the State Bar’s staff to determine its process for evaluating vendors and entering into contracts for the bar exam, including requirements associated with competitive bidding exemptions.

Source: Analysis of state law and audit workpapers.

Assessment of Data Reliability

The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of the computer‑processed information that we use to support our findings, conclusions, or recommendations. In performing this audit, we relied on the State Bar’s case management data. To evaluate these data, we reviewed existing information about the data, interviewed staff knowledgeable about the data, performed electronic testing of the data, and traced a sample of the data to supporting documents. We found that these data were sufficiently reliable for the purposes of this audit.