Appendix

SCOPE AND METHODOLOGY

The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to perform an audit related to EDD’s privacy protection practices when mailing documents to its customers, as well as other audit objectives. The table below outlines the Audit Committee’s objectives and our methods for addressing them.

Audit Objectives and the Methods Used to Address Them
AUDIT OBJECTIVE		METHOD
1	Review and evaluate the laws, rules, and regulations significant to the audit objectives.	Researched and reviewed relevant laws, rules, regulations, and policies.
2	Determine whether EDD’s policies and procedures for protecting customers’ personal information comply with applicable state and federal laws and state policy.	Obtained and reviewed EDD’s information security policies and its policy and procedures manuals for Disability and Unemployment.
3	Determine whether EDD has been mailing documents to its customers since 2015 that contain personal information and, if so, determine the following:	Interviewed key staff and reviewed records related to EDD mailing documents to claimants that contain SSNs and other personal information and its efforts to remove SSNs from mailed documents. Obtained inventory lists and document templates for documents that EDD mailed or received by mail with SSNs, or for documents from which it had removed or otherwise mitigated its use of SSNs from 2015 through 2018, and determined whether EDD included other personal information on its Disability and Unemployment documents. Selected 21 documents that EDD mails to Disability or Unemployment claimants—including documents both with and without SSNs—and evaluated the documents’ functions, how frequently EDD mailed the documents, its reasons for including full SSNs on the documents, and whether it was required or permitted to mail the documents with SSNs. Interviewed key staff and reviewed records related to EDD’s processes for ensuring that its documents include only necessary personal information. Obtained from EDD’s IT branch the number of registered users for its Disability and Unemployment online systems and the number of those users requesting to receive online communication from EDD.
	a. EDD’s reasons for mailing documents to its customers that contained full SSNs or other personal information rather than using other alternative methods, such as redacting the SSNs.
	b. To the extent possible, the number of individuals who requested to receive information online only but were mailed documents containing full SSNs or other personal information.
4	Determine whether EDD provides, or plans to provide, alternatives to mailed documents, including providing online communication. If so, to the extent possible, evaluate the effectiveness of those alternatives to increase customer privacy.	We did not identify any additional issues that are significant to the audit.
5	Determine the number of complaints EDD has received from its customers about receiving documents through the mail that contain SSNs, including any complaints related to identity theft. Determine whether EDD adequately responded to those complaints.	Interviewed key staff and reviewed policy manuals to determine EDD’s methods for processing complaints. Obtained and analyzed complaint data from EDD’s database of electronic communications with claimants to assess the number and nature of complaints. Due to limitations associated with the various complaint channels available to claimants and EDD’s method for recording those complaints, EDD was unable to determine a precise number of claimants who complained to it regarding SSNs on mailed documents. Further, the records EDD maintains in its database of electronic communications do not always match the category of the complaint and also contain multiple free form data fields, which present challenges when searching and filtering the data. Nevertheless, we reviewed numerous complaints. Reviewed a selection of reports that EDD provided that summarized information security incidents from 2015 through 2018. When we requested clarification of certain issues arising from our review of those documents, EDD was unable to locate some of its internal documentation of information security incidents. It instead provided us with records from the California Compliance and Security Incident Reporting System (Cal‑CSIRS), to which it reports such incidents. We found the information available in the Cal‑CSIRS records to be less detailed than what EDD captured in its internal reports and consequently insufficient for the purposes of our review. Due to this limitation, the information we present regarding EDD’s information security incidents is not meant to reflect EDD’s information security incidents in total.
6	Evaluate EDD’s efforts since 2015 and plans to better protect personal information of its customers and determine the costs and timelines of these efforts. Determine whether any other resources or low-tech or temporary options are available to resolve this issue.	Interviewed relevant staff and obtained documents related to EDD’s previous efforts to increase its protection of claimants’ privacy and its future plans to do so. Reviewed planning documents related to EDD’s modernization project and pertinent approved budget change proposals. Discussed with key staff our proposed interim solutions so that we could determine the solutions’ feasibility and obtain estimated implementation costs and timelines. We determined that certain solutions—such as manually redacting SSNs from EDD’s mailings—would be impractical because of the cost, effort, and risks involved. We present the remaining solutions in the Audit Results.
7	Review and assess any other issues that are significant to the audit.	Did not identify any additional issues that are significant to the audit.

Source: Analysis of the Audit Committee’s audit request number 2018-129, as well as information and documentation identified in the column titled Method.

Assessment of Data Reliability

In performing this audit, we obtained electronic data from EDD related to its metered mail and online communications, including complaints. The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of any computer‑rocessed information we use to support our findings, conclusions, or recommendations. We found the data related to metered mail to be reasonable; however, we found limitations with the online communications data, which we describe in the Scope and Methodology table. To evaluate these data, we performed electronic testing of the data and interviewed key staff knowledgeable about the data. We did not perform accuracy or completeness testing of these data so they are of undetermined reliability for our audit purposes. Although these determinations may affect the precision of the numbers we present, there is sufficient evidence in total to support our findings, conclusions, and recommendations.