Appendix A

2020 EDD Call Data

The Audit Committee asked us to report on certain trends in calls to EDD's call center. Specifically, the Audit Committee asked us to determine trends in the volume of calls received, the time it takes EDD to respond to callers, the percentage of callers connected to an agent, and the number of calls prematurely disconnected. Table A presents call data that EDD provided us for 2020, including the trends the Audit Committee requested. However, EDD does not track the number of calls in which the caller was disconnected. Instead, we present the number of unanswered calls. We determined the number of unanswered calls by combining the number of calls blocked from entering the system, calls that entered the system but were then deflected because an agent was not available to answer, and calls that the caller abandoned before an agent answered.⁷Blocked and deflected calls generally occur during periods of high call volume. Blocked calls were caused by a technological limitation in EDD's old phone system that limited the number of calls the system could handle at once. Deflected calls could occur in EDD's old phone system and can still occur in its new VCC system when EDD does not have enough agents to answer the number of incoming calls and wait times exceed established limits. For blocked and deflected calls, the phone system plays a recorded message telling the caller that EDD cannot take their call and to call back later.

Table A

EDD Call Center Metrics for 2020

Month*	Average Weekly Unique Callers^†	Total Calls	Calls Answered by EDD Agents	Percentage of Calls Answered^‡	Unanswered Calls^§	Percentage of Calls Unanswered^‡	Calls to Self-Service Line	Percentage of Calls to Self-Service Line^‡	Weekly Average Full-Time Equivalent Agents on Phone^II	Average Hold Time (minutes)
January (12/29/19 – 2/01/20)	120,080	2,817,338	189,801	6.7%	1,627,879	57.8%	1,021,259	36.3%	159	9.5
February (2/02/20 – 2/29/20)	121,365	1,435,635	180,401	12.6	609,946	42.5	665,215	46.3	172	2.8
March (3/01/20 – 3/28/20)	301,981	5,785,032	148,268	2.6	3,449,859	59.6	2,205,025	38.1	199	5.9
April^# (3/29/20 – 5/02/20)	1,127,561	61,592,966	279,449	0.4	52,813,826	85.8	8,527,199	13.8	222	20.6
May (5/03/20 – 5/30/20)	1,299,632	53,052,337	533,128	1.00	47,708,529	89.9	4,845,510	9.1	574	28.2
June (5/31/20 – 6/27/20)	1,118,034	46,220,542	519,394	1.1	41,472,258	89.7	4,237,133	9.2	599	34.9
July (6/28/20 – 8/01/20)	1,048,233	50,251,351	697,132	1.4%	44,161,328	87.9%	5,401,211	10.8%	669	31.6
August (8/02/20 – 8/29/20)	720,810	17,271,613	775,825	4.5	13,359,085	77.4	3,143,201	18.2	931	32.8
September (8/30/20 – 10/03/20)	640,703	11,031,294	1,143,254	10.4	6,404,147	58.1	3,490,532	31.6	1,018	20.0
October (10/04/20 – 10/31/20)	859,210	3,649,193	230,301	6.3	1,735,764	47.6	1,684,715	46.2	1,153	16.3

Source: EDD reports on call data.

* EDD's call data is summarized weekly, leading to some months with 4 weeks and some with 5 weeks.

^† EDD's call data records the number of unique callers in a given week. Therefore, this column records the average weekly callers for each month.

^‡ Percentages do not total to 100 percent due to rounding.

^§ Claimants whose calls were blocked, deflected, or abandoned ultimately did not speak to an agent; therefore, we have summarized these metrics under the single heading of unanswered calls.

^II EDD agents perform both on- and off-phone work in a given week; therefore, the call data records the number of full-time equivalent staff who answered phones each week.

^# EDD implemented the first version of its new VCC phone system in April 2020.

Appendix B

Scope and Methodology

The Audit Committee directed the State Auditor in September 2020 to conduct an emergency audit of EDD's response to COVID-19. Additionally, state law authorizes our office to establish a program to audit and issue reports with recommendations to improve any state agency or statewide issue that we identify as being at high risk for the potential of waste, fraud, abuse, and mismanagement or that has major challenges associated with its economy, efficiency, or effectiveness. In August 2020, we designated the State's management of federal funds related to Coronavirus Disease 2019 (federal COVID‑19 funds) as a high‑risk statewide issue, and EDD as a state agency responsible for a portion of that statewide issue. For these reasons, we performed this audit of EDD's UI program. The table below lists the audit objectives and the methods we used to address them.

Table B

Audit Objectives and the Methods Used to Address Them

AUDIT OBJECTIVE		METHOD
1	Assess the reasons for backlogged unemployment insurance claims (claims) and the effectiveness of EDD's efforts and time frames for eliminating the backlog, including but not limited to, technological issues and state or federal laws that have contributed to the delay or that prevent EDD from processing claims faster.	Reviewed federal and state law and documents and interviewed staff to assess EDD's UI program. Interviewed staff and obtained documentation, including EDD policies, procedures, and reports, to determine the size of the backlog and the reasons claims became backlogged from March through the end of September 2020. Analyzed documentation to determine whether EDD was successful at reducing the claims backlog. Determined what mechanisms and strategies EDD employed to reduce the backlog and the rate of manual staff intervention in claims processing. Analyzed documentation provided by EDD to determine what work EDD temporarily suspended, including making claimant eligibility determinations, and how that delayed workload would affect EDD's ability to process backlogged claims or conduct its regular business. Interviewed staff and analyzed documentation to determine whether technological issues contributed to delays in claim processing.
2	Evaluate the effectiveness of EDD's actions to improve call center performance and response time. This evaluation should also include EDD's hiring, on-boarding, and training efforts to increase call center staffing levels.	Interviewed staff and obtained documentation of EDD's actions to improve call center performance. Reviewed staffing documentation to determine how many staff EDD had redirected from its other branches, borrowed from other state agencies, contracted for, and hired from January through October 2020. Reviewed training materials and plans to determine the timing and content of training EDD provided to its new call center staff. Analyzed callback data and call transfer data to evaluate the effectiveness of new call center staff. Reviewed research articles and reports regarding call center operations to identify best practices and determined whether EDD had implemented those best practices for its call centers. Evaluated EDD's draft Economic Resilience Plan to determine whether EDD had policies and procedures in place to respond to an economic downturn and whether the plan included emerging recession planning practices. Assessed the extent to which EDD addressed past audit findings concerning the UI program.
3	Determine the magnitude of EDD's claims workload, including the number and percentage of claims that were approved, denied, pending, and backlogged since the beginning of the pandemic.	Analyzed EDD reports to determine the number of claims EDD received and the percentages it approved, denied, or had pending in the backlog as well as the age of claims in the backlog as of December 9, 2020. To determine the percentage increase in claims during the pandemic, we compared the number of claims received in 2010 and 2019 to those received in 2020. Interviewed staff and obtained documentation to determine how EDD resolved claims, and whether it complied with relevant state and federal requirements. Reviewed EDD's reports pursuant to AB 107 with respect to the number of denied claims and found EDD's calculation approach to be reasonable. As of early January 2021, EDD reported the number of denied claims since March 2020 to be about 675,000, or about 4 percent of all claims processed.
4	Assess EDD's call center capacity and determine trends in the volume of calls received, the time it takes EDD to respond to callers, the percentage of callers connected to a representative, and the number of calls where the caller was disconnected from the call.	Interviewed staff and obtained documentation of EDD's old phone system and its current VCC phone system to determine the capacity and features of each. Analyzed weekly call data reports to determine trends in the volume of calls that EDD received, the time it takes EDD to respond to callers, the percentage of unique callers connected to an agent, and the number of unsuccessful calls. Unsuccessful calls include calls that were blocked from the entering the system, calls that were deflected because no agents were available, and calls that were abandoned by the caller. EDD call data reports do not track the number of calls that were disconnected.
5	Assess the technological infrastructure to determine if it is delaying or preventing EDD from processing claims faster.	Interviewed staff and reviewed documentation, including system performance reports, documents tracking EDD's progress on implementing strike team recommendations, IT Plan of Action and Milestones, correspondence between department staff and the Office of Digital Innovation, and various Benefits Systems Modernization status documents.

Source: Analysis of state law, planning documents, and information and documentation identified in the table column titled Method.

Assessment of Data Reliability

The U.S. Government Accountability Office, whose standards we are statutorily obligated to follow, requires us to assess the sufficiency and appropriateness of computer-processed information we use to support our findings, conclusions, or recommendations. In performing this audit, we relied on electronic data files and summary reports from information systems that we obtained from EDD's UI and IT branches. To evaluate the data, we reviewed existing information about the data, interviewed staff knowledgeable about the data, and assessed documentation to validate general details about the data. In addition, we reviewed the query that EDD uses to calculate its number of backlogged claims to better understand how the department calculates its reported numbers. EDD was unable to provide a complete description of the query. Further, in light of the short timeframe of this emergency audit, we did not perform detailed testing of the data we relied on. Consequently, we found the data to be of undetermined reliability. Although we recognize this determination may affect the precision of the numbers we present, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations.