The Governor of California
President pro Tempore of the Senate
Speaker of the Assembly
State Capitol
Sacramento, California 95814
Dear Governor and Legislative Leaders:
This letter report presents a summary of the results of the California State Auditor’s (State Auditor) assessments of the reliability of data in a wide variety of the State’s information technology systems used by the State Auditor for the purposes of its audits. Since October 2008, we have issued biennial reports that address the reliability of the data from the State’s systems we tested as part of audits issued during calendar years 2006 through 2013. The reliability of the data from the State’s systems tested during audits that were issued in 2014 and 2015 is the subject of this report. This report also summarizes the results of our high risk audit concerning weaknesses in the controls over the State’s information systems.
Definitions Used in
Data Reliability Assessments
Sufficiently Reliable Data—Based on audit work, an auditor can conclude that the likelihood of significant errors or incompleteness is minimal and that using the data would not lead to an incorrect or unintentional message given the research question and intended use of the data.
Not Sufficiently Reliable Data—Based on audit work, an auditor can conclude that results indicate significant errors or incompleteness in some or all the key data elements, and that using the data could lead to an incorrect or unintentional message, given the research question and the intended use of the data.
Data of Undetermined Reliability—Based on audit work, an auditor can conclude that use of the data may or may not lead to an incorrect or unintentional message, given the research question and intended use of the data.
Source: U.S. Government Accountability Office.
The U.S. Government Accountability Office (GAO), whose standards we are statutorily required to follow, requires us to assess and report on the reliability of computer-processed information that we use to support our audit findings, conclusions, and recommendations. Data reliability refers to the accuracy and completeness of the data, given our intended purposes for the data’s use. The GAO defines the three possible assessments we can make—sufficiently reliable data, not sufficiently reliable data, and data of undetermined reliability (see the text box for definitions). In assessing data reliability, we take several factors into consideration, including the degree of risk involved in the use of the data and the strength of corroborating evidence. A single system may have different assessments, for example, because data that we use for one audit purpose is accurate and complete, whereas data from the same system needed for a separate purpose are not. The State uses these data in many ways, which include reporting on its programs, processing payroll and personnel transactions, and managing the State’s finances. Although we disclosed these data reliability assessments and any data limitations we identified in the audit reports that we issued during 2014 and 2015, this report is intended to call attention both to areas of concern, where important data are not always reliable, and to instances in which information has been reliable. Further, this report highlights our finding that many state entities have weaknesses in their controls over information security. These weaknesses leave some of the State’s sensitive data vulnerable to unauthorized use, disclosure, or disruption.
Many Systems Had Reliable Data for the Purposes of the Audits
In performing 78 data reliability assessments for state systems, we determined that the data were sufficiently reliable in 13 assessments. Therefore, for these assessments, we were able to use the data to support our audit findings, conclusions, and recommendations and to quote the data in our audit reports without qualifications about the accuracy or completeness of the information. For example, we found no issue in the Bureau for Private Postsecondary Education’s data as maintained in the Department of Finance’s California State Accounting and Reporting System and used it to determine the beginning balance, ending balance, and total revenues and expenditures for the Student Tuition Recovery Fund for fiscal years 2008–09 through 2012–13. In addition, we found no issue in the Department of Housing and Community Development’s Cumulative Propositions 46 and 1C Bond Awards and used this data to identify the total number and amount of awards by program as of December 31, 2013. Also, we determined that the State Bar of California’s Discipline Case Tracking system was sufficiently reliable to calculate the number of complaints open and closed in the intake unit, determine the average case processing time, and total the caseload and the number of cases backlogged from 2009 to 2014. Finally, we found the State Controller’s Office (State Controller) Budgetary/Legal Basis System sufficiently reliable to determine the California Department of Resources Recycling and Recovery’s beverage program’s revenues, expenditures, and fund balances for fiscal years 2010–11 through 2013–14.
Many Systems Were Not Sufficiently Reliable for the Purposes of the Audits
For 22 data reliability assessments, we concluded that the data were not sufficiently reliable. Whenever we include these data in our reports, we make the limitations of the data known so that incorrect or unintentional conclusions would not be drawn. For example, we obtained the Department of Toxic Substances Control’s Cost Recovery Billing System to determine the amount of unbilled and billed but uncollected cleanup costs. However, we disclosed in our audit report that the department acknowledges the unreliability of the data contained in its billing system, and has little confidence that the billing statuses of its outstanding costs are correct. Consequently, we reported that the Cost Recovery Billing System was not sufficiently reliable for the purpose of the audit. In addition, we obtained the State Controller’s California Leave Accounting System (leave accounting system) to determine the amount of leave employees should have received. While performing electronic testing of the leave accounting system, we found errors in more than 14,000 employee records. In addition, we traced 55 haphazardly selected time sheets of employees who worked alternate workweek schedules to supporting documents and found 11 errors. Therefore, we disclosed in our audit that we determined that these data were not sufficiently reliable.
In some circumstances, we recommended that the audited agency take corrective action. To improve the accuracy of information in the leave accounting system and to ensure that agencies do not improperly credit employees with leave in the future, we made several recommendations to the State Controller. First, we recommended that the State Controller implement additional controls by June 2015 to prevent the leave accounting system from processing the types of inappropriate transactions we identified in our statewide electronic analysis. For example, we suggested that it could develop cost‑effective controls in the leave accounting system that would prevent employees from receiving annual leave and sick leave during the same pay period. We also recommended that the State Controller work with the California Department of Human Resources (CalHR) to establish procedures by January 2015 for updating the criteria it uses to produce the monthly exception reports to ensure that the criteria reflect changes in state law and collective bargaining agreements. Further, using criteria provided by CalHR, we recommended that the State Controller develop monthly exception reports that identify transactions in the leave accounting system that are inconsistent with the guidelines established in state law and collective bargaining agreements, such as instances in which state employees receive too many personal holidays or too much holiday credit. Finally, we recommended that by June 2015 the State Controller begin providing each state agency’s human resources management with the transactions identified in the exception reports for review and correction as necessary. In response to our recommendations, the State Controller began identifying and analyzing potential system enhancements, worked with CalHR to establish procedures for updating the criteria it uses to produce monthly exception reports, and started producing and distributing some exception reports.
We Were Unable to Determine the Reliability of Data for Some Audits
For 43 data reliability assessments, we concluded that the data were of undetermined reliability. In many cases, the determination that data were of undetermined reliability arose from our decision to limit testing due to impracticality or the prohibitively high cost of fully testing a database. This was the case when source documents were housed at numerous locations throughout the State, or when the system was primarily paperless, and thus, hard‑copy documentation was not available for review.
For instance, we determined that data from the California Department of Justice’s (Justice) Armed Prohibited Persons System and Mental Health Firearms Prohibition System were of undetermined reliability for the purposes of identifying daily backlog, forecasting Justice’s completion of the historical backlog, and identifying trends in court and mental health facility reporting. We were unable to perform accuracy and completeness testing of these data because the source documents required for this testing are stored by various entities such as mental health facilities, courts, or firearm retailers located throughout the state, making such testing cost‑prohibitive. In addition, we performed an audit regarding the State’s compliance with federal and state web accessibility standards in which we used data from five systems provided by three state government entities—California Health Benefit Exchange, California Community Colleges, and the Franchise Tax Board. We concluded that the data were of undetermined reliability for the purpose of identifying a selection of web accessibility defects and user complaints. We did not perform accuracy and completeness testing of these data because the systems are paperless and hard‑copy source documentation was not available for review. Alternatively, following GAO guidelines, we could have reviewed the adequacy of selected system controls that include general and application controls. However, we did not conduct these reviews because this audit involved five such paperless systems across three departments and to do so for each would have been cost‑prohibitive.
Many State Entities Have Poor Controls Over Their Information Systems, Putting Some of the State’s Most Sensitive Information at Risk
In addition to the concerns we noted in performing the data reliability assessments previously discussed, we also identified other weaknesses that could compromise the information systems the State uses to perform its day‑to‑day operations. We disclosed these weaknesses in our August 2015 report titled High Risk Update—Information Security: Many State Entities’ Information Assets Are Potentially Vulnerable to Attack or Disruption (Report 2015-611). Specifically, the California Department of Technology (technology department) is responsible for ensuring that state entities that are under the direct authority of the Governor (reporting entities) maintain the confidentiality, integrity, and availability of their information systems and protect the privacy of the State’s information. As part of its efforts to protect the State’s information assets, the technology department requires reporting entities to comply with the information security and privacy policies, standards, and procedures it prescribes in Chapter 5300 of the State Administrative Manual (security standards)1 . However, we found that the majority of reporting entities—including some that maintain sensitive or confidential information—had yet to achieve full compliance with the security standards.
Specifically, we performed compliance reviews of selected information security requirements at five reporting entities and found that each had deficiencies. The reporting entities we reviewed perform a variety of important roles within state government, from regulatory to enforcement activities. We focused our review of security standards on three key controls areas that form the foundation of an effective information security control structure: information asset management, risk management, and information security program management. We also reviewed the two control areas related to a reporting entity’s ability to respond to incidents and disasters: information security incident management and technology recovery. Figure 1 describes these five control areas. These control areas comprise 17 of the 64 sections of the security standards.
Figure 1
Five Key Control Areas of Information Security With Which the California Department of Technology Requires Reporting Entities to Comply
Source: California State Auditor’s (State Auditor) assessment of the information security standards outlined in Chapter 5300 of the State Administrative Manual (security standards).
Note: The State Auditor focused its review on the five key control areas above, which include 17 of the 64 sections of the security standards.
Although all five reporting entities maintain different types of sensitive data, each had deficiencies in their ability to protect such data, as Table 1 shows. In fact, only one achieved full compliance in any of the areas we tested. All five reporting entities had not met or had only partially met the requirements to establish and maintain an inventory of their information assets. Four had not met or had only partially met the requirements associated with two control areas: managing the risks to their information assets and developing a comprehensive information security program to address their risks. In addition, none had fully met the requirements related to developing an incident response plan for handling information security incidents such as malicious cyberattacks and developing a technology recovery plan for addressing unplanned disruptions due to natural disasters or other causes. However, two reporting entities were mostly compliant in these two areas.
Reporting Entity |
Entity Description |
Collects, Stores, or Maintains |
Information Asset Management | Risk Management | Information Security Program Management | Information Security Incident Management | Technology Recovery | ||
|---|---|---|---|---|---|---|---|---|---|
| Personal Information or Health Information Protected by Law | Confidential Financial Data | Other Sensitive Data | |||||||
A |
Provides critical state services | Yes | Yes | Yes | Partially compliant | Partially compliant | Partially compliant | Partially compliant | Partially compliant |
B |
Administers federal and state programs |
Yes | No | No | Partially compliant | Fully compliant | Fully compliant | Mostly compliant | Mostly compliant |
C |
Oversees an entitlement program | Yes | Yes | Yes | Partially compliant | Partially compliant | Partially compliant | Partially compliant | Partially compliant |
D |
Performs enforcement activities | Yes | No | Yes | Not compliant | Partially compliant | Not compliant | Not compliant | Partially compliant |
E |
Manages critical state resources | Yes | No | Yes | Partially compliant | Not compliant | Partially compliant | Mostly compliant | Mostly compliant |
Fully compliant: The reporting entity was fully compliant with all the requirements in Chapter 5300 of the State Administrative Manual (security standards) we tested for the control area.
Mostly compliant: The reporting entity had attained nearly full compliance with all of the security standards we tested for the control area.
Partially compliant: The reporting entity had made measurable progress in complying, but had not addressed all of the security standards we tested for the control area.
Not compliant: The reporting entity had not yet addressed the security standards we tested for the control area.
Similarly, our survey of reporting entities showed that most have yet to achieve full compliance with the State’s information security requirements. Specifically, we surveyed 101 reporting entities and asked them to designate their compliance status with each of the 64 sections of the security standards. However, only four of the 77 survey respondents that completed the entire survey asserted that they had fully complied with all of the security standards; the remaining 73 reporting entities reported various levels of noncompliance with the requirements. As Figure 2 shows, for each of the five control areas, at least 49 of the 77 survey respondents stated that they had yet to achieve full compliance with the security standards. The survey respondents reported that they had made the most progress toward achieving compliance with the information security incident management and technology recovery requirements: more than 70 percent of respondents indicated that they were mostly or fully compliant with these requirements. Conversely, nearly half of the survey respondents indicated that they had not or had only partially met the requirements for risk management. Finally, 22 of the 77 survey respondents stated that they did not expect to reach full compliance with the information security standards until 2018 or later, with 13 indicating that they would be out of compliance until at least 2020.
Figure 2
Reporting Entities’ Levels of Compliance With Select Information Security Control Areas for 2014, According to Their Survey Responses
Source: California State Auditor’s analysis of survey responses from 77 reporting entities.
Green = Fully compliant: The reporting entity asserted it was fully compliant with all the requirements in Chapter 5300 of the
State Administrative Manual (security standards) for the control area.
Yellow = Mostly compliant: The reporting entity asserted it had attained nearly full compliance with all of the security standards
for the control area.
Orange = Partially compliant: The reporting entity asserted it had made measurable progress in complying, but had not addressed
all of the security standards for the control area.
Red = Not compliant: The reporting entity asserted it had not yet addressed the security standards for the control area.
Because our survey included self‑reported information and our five compliance reviews focused only on select information security controls, the reporting entities’ information security controls may have additional deficiencies that we did not identify. Alternatively, some reporting entities may have compensating information security controls that help mitigate some of the risks associated with not being fully compliant. Nevertheless, the weaknesses we identified could compromise the confidentiality, integrity, and availability of the information systems these reporting entities currently use to perform their day‑to‑day operations. As a result of the outstanding weaknesses in reporting entities’ information system controls, we determined that some of the State’s information, and its critical information systems, are potentially vulnerable and continue to pose an area of significant risk to the State.
Appendix
Summary of Reliability Assessments for Audits Issued in 2014 and 2015
The following table summarizes selected information from the data reliability assessments contained on the State Auditor’s website. Additional information is also available on the website which further describes any limitations we identified in the data. Although we recognize that these limitations may impact the precision of the numbers we presented in our reports, there was sufficient evidence in total to support our audit findings, conclusions, and recommendations.
| Agency* | Information System† | Magnitude of Data | Reliability for Audit Purposes‡ | Agency purpose of data§ | Audit Number |
|---|---|---|---|---|---|
| BUSINESS, CONSUMER SERVICES AND HOUSING AGENCY | |||||
| BPPE’s data as maintained in Department of Finance’s (Finance) California State Accounting and Reporting System (CALSTARS) | CALSTARS contained more than $10.5 million per fiscal year in total revenues and nearly $1.1 million per fiscal year in total expenditures for the Student Tuition Recovery Fund balance for fiscal years 2008–09 through 2012–13. | Yes | An automated organization and program cost‑accounting system to accurately and systematically account for all revenue, expenditures, receipts, disbursements, and property of the State. | 2013-045 | |
| Schools Automated Information Link (SAIL) | SAIL contained almost 22,000 records tracking license application information; more than 12,000 records related to Student Tuition Recovery Fund claims processing information; and almost 17,000 records related to complaints and investigations information. | No, Undeterminedll | The BPPE uses the SAIL system to track Student Tuition Recovery Fund assessments, claims, and inspections. | 2013-045 | |
| Department of Housing and Community Development’s data as maintained in Finance’s CALSTARS | The CALSTARS data contained nearly 223 thousand expenditure records for the period from March 1, 2012, through March 31, 2014. | Yes | An automated organization and program cost‑accounting system to accurately and systematically account for all revenue, expenditures, receipts, disbursements, and property of the State. | 2014-037 | |
| Cumulative Proposition 46 and Proposition 1C Bond Awards Report (bond awards reports) | The Proposition 46 Bond Awards Report contains 1,944 records for awards between March 1, 2012, and December 31, 2013. Further, the Proposition 1C Bond Awards Report contains 1,230 records for awards between March 1, 2012, and December 31, 2013. | Yes | The bond awards reports are a point‑in‑time status of all Proposition 1C and Proposition 46 funds and are the basis of the information reported annually to the Legislature and the information provided on the California Bond Accountability website. | 2014-037 | |
| CORRECTIONS AND REHABILITATION, CALIFORNIA DEPARTMENT OF | |||||
| Strategic Offender Management System (SOMS) | SOMS contains offender management and electronic records for more than one million offenders. | No# | SOMS is an integrated offender management system and electronic records management system. | 2013-120 | |
| Tests of Adult Basic Education (TABE) Master File Access Database | Contains more than 900,000 records of TABE reading scores from June 1964 to February 2014 using inmates’ CDC numbers for identification. | No# | The TABE Master File Access Database is used to track inmates’ TABE reading scores. | 2013-120 | |
| Corrections’ data as maintained in the State Controller’s Office (State Controller) Employment History Database |
We received nearly 1.4 million employment records for Corrections’ employees. | Undetermined | Electronic database containing personnel records for state employees. | 2014-117 | |
| Corrections’ data as maintained in the State Controller’s Uniform State Payroll System | The data we received from the Uniform State Payroll System included a total of more than 31 million records representing payroll transactions for the period July 2010 through September 2014. | Undetermined | The State Controller uses the Uniform State Payroll System to process the State’s payroll and personnel transaction documents. | 2014-117 | |
| Vehicle Home Storage Permit Database | The data included more than 2,000 vehicle home storage permits in effect for Corrections’ employees during fiscal year 2012–13 and more than 1,600 permits in effect during fiscal year 2013–14. | Yes | To track vehicle home storage permits issued to Corrections’ employees. | 2014-117 | |
| CALIFORNIA ENVIRONMENTAL PROTECTION AGENCY | |||||
| California Department of Resources Recycling and Recovery (CalRecycle) |
CalRecycle’s data as maintained by the State Controller’s Budgetary/Legal Basis System | The Budgetary/Legal Basis System recorded an ending balance for fiscal year 2013–14 of more than $312 million for the five funds that support the Beverage Container Recycling Program—the Beverage Container Recycling Fund, the Glass Processing Fee Account, the Penalty Account, the Bimetal Processing Fee, and the PET Processing Fee Account. | Yes | The Budgetary/Legal Basis System tracks financial data, indexes, funds, etc. on a budgetary/legal basis. It is manually updated based on data from numerous sources. | 2014-110 |
| Cost Recovery Billing System | The Cost Recovery Billing System contained more than $193 million in unbilled or uncollected costs from July 1987 through March 10, 2014. | No | To record payments related to cost recovery | 2013-122 | |
| EnviroStor | EnviroStor contained data on more than 11,000 cleanup sites as of July 2015. | Undetermined | To record cleanup and permitting site and facility information. |
2013-122 | |
| Federal and state statutes of limitations tracking spreadsheet | The statute of limitations tracking spreadsheet contained information on more than 600 projects as of January 2014. | Yes | To provide information needed to establish when the statute of limitations expired or will expire for a project. | 2013-122 | |
| CALIFORNIA GOVERNMENT OPERATIONS AGENCY | |||||
| Department of General Services |
State Contract and Procurement Registration System (SCPRS) | SCPRS contained more than 93,000 contracts procured from July 1, 2012, through June 30, 2013. | Undetermined | SCPRS was established in 2003 as a centralized database of information on state contracts and purchases over $5,000. | 2013-115 |
| Franchise Tax Board | ClearQuest | The database contained more than 8,900 entries. | Undetermined | The database is used to track potential enhancements, potential changes, and maintenance efforts in CalFile, the Franchise Tax Board’s electronic tax filing application. Once a change was approved, ClearQuest was used to track defects regarding the change. | 2014-131 |
| CALIFORNIA HEALTH AND HUMAN SERVICES AGENCY | |||||
| Automated Survey Processing Environment Complaints/Incidents Tracking System (ACTS) | ACTS included information related to more than 366,000 complaints against facilities. | No, Undeterminedll | Enables state agencies to implement information‑based administration of the health care facilities under their supervision. | 2014-111 | |
| Professional Certification Branch’s investigation section’s Case Management Spreadsheet | The spreadsheet contained over 3,000 records describing complaints against certified paraprofessionals. | No | To record and monitor the status of investigations. | 2014-111 | |
| Electronic Licensing Management System (ELMS) | The ELMS data included information for the more than 50 state‑owned facilities of the State Facilities Unit. | Undetermined | Records health service providers’ applications, issues licenses, generates license renewal notices, determines license fees, issues and tracks state enforcement actions, and generates management reports. | 2014‑111 | |
| California State Accounting and Reporting System On‑Line Reporting Environment (CORE) | The data contained more than $4 million in expenditures from March 29, 2009, through June 30, 2013, for the California Department of Public Health’s diabetes prevention. |
Yes | An automated organization and program cost‑accounting system to accurately and systematically account for all revenue, expenditures, receipts, disbursements, and property of the State. | 2014-113 | |
| Department of Developmental Services (Developmental Services) | Cost Recovery System (CRS) | The data contained 730 consumers who received services at some point from Developmental Services through the Parental Fee Program in fiscal year 2013–14. | No | Developmental Services’ Parental Fee Program uses CRS to maintain payment and billing information and information on fee assessments for client accounts. | 2014-118 |
| Fiscal Intermediary Access to Medi‑Cal Eligibility system (FAME) | FAME contained nearly 750 million monthly beneficiary eligibility records for the period July 2008 through December 2013. | No, Undeterminedll | To provide Medi‑Cal eligibility data for the purpose of Medi‑Cal claims adjudication. | 2013-119, 2013-125 | |
| Master Provider File (provider eligibility database) | As of March 2014, the provider eligibility database contained eligibility information for nearly 4,800 substance use disorder services provider sites. | No | To track all providers that provide substance use disorder services. | 2013-119 | |
| Provider suspension spreadsheet | The provider suspension spreadsheet contained 235 suspensions as of April 2014. | Undetermined | To track suspended providers. | 2013-119 | |
| Short‑Doyle Medi‑Cal ADP Remediation Technology System (program billing application) | As of March 2014, the program billing application contained claim information for nearly 26 million services that Drug Medi‑Cal Treatment Program providers submitted for payment since the system’s implementation in January 2010. | No, Undeterminedll | To track the adjudicated Drug Medi‑Cal Treatment Program claim results and payments. | 2013-119 | |
| California Dental Medicaid Management Information System (CD‑MMIS) | The CD‑MMIS data contained more than 22 million dental procedures authorized for payment in 2012. | No, Undeterminedll | CD‑MMIS is used to track paid or denied dental claims and for claims processing and reporting. | 2013-125 | |
| California Medicaid Management Information System (CA‑MMIS) | The CA‑MMIS data contained information for more than 772,000 dental visits in 2012. | No, Undeterminedll | CA‑MMIS is used for paying encounters, claims processing, and reporting. | 2013-125 | |
| Administrative Claiming Local and School Services Branch’s Medi‑Cal Administrative Activities Invoice Database (invoice database) | The invoice database contained more than 32,000 records relating to quarterly claims submitted by local governmental agencies and local educational consortia. | Yes | To track invoices that Health Care Services receives, generates reports for stakeholders, and tracks when invoices are forwarded to other departments. |
2014-130 | |
| Management Information System/Decision Support System (MIS/DSS) | The MIS/DSS data contains more than 245,000 claims for September 2011. | Undetermined | To track the claims paid for use in preparing the quarterly CMS‑64 report. | 2014-130 | |
| School‑Based Medi‑Cal Administrative Activities Interim Claiming and Reasonableness Test Criteria Tracker Database | The data contains more than 5,000 records as of February 2015. | No | To track the Reasonableness Test Criteria invoices and run reports as needed. | 2014-130 | |
| Medi‑Cal Managed Care Office of the Ombudsman’s AT&T Call Management System |
The AT&T Call Management system handles 25,000 calls per month. | Undetermined | AT&T Call Management data is used to track trends related to the resolution of calls the Medi‑Cal Managed Care Office of the Ombudsman receives in its call center. It uses this information to train staff on current issues and trends. | 2014-134 | |
| Sex Offender Commitment Program Support System (SOCPSS) | SOCPSS contained records of nearly 56,000 cases referred by Corrections, more than 38,000 evaluations performed prior to commitment to State Hospitals, and more than 14,000 evaluations performed post‑commitment to State Hospitals. | Yes, Noll | State Hospitals uses SOCPSS as a single repository to track all Sexually Violent Predators (SVP), all new offenses and subcategories of offenses required by SB 1128 and Jessica’s Law, and all case activities throughout the SVP commitment cycle. | 2014-125 | |
| State Hospitals’ data as maintained in the State Controller’s Uniform State Payroll System | The Uniform State Payroll System contained more than 38.7 million payroll records for the period of July 2009 through September 2014. | Undetermined | The State Controller uses the Uniform State Payroll System to process the State’s payroll and personnel transaction documents. | 2014-125 | |
| CALIFORNIA LABOR AND WORKFORCE DEVELOPMENT AGENCY | |||||
| Employment Development Department | California Unemployment Insurance Appeals Board’s Enhanced California Appeals Tracking System (eCATS) | The eCATS database contained nearly 1.5 million first level decisions between July 1, 2010, and April 23, 2014. | Undetermined | California Unemployment Insurance Appeals Board uses the eCATS system to track and process first‑level appeals and when necessary second‑level appeals. In addition, it is used for management reports and interfaces with three other applications. | 2014-101 |
| OTHER DEPARTMENTS, OFFICES AND UNIVERSITIES | |||||
| JIRA | JIRA contains more than 4,800 records related to change requests. | Undetermined | JIRA is a tool used to track and document the development process for the California Community Colleges’ (Community Colleges) OpenCCC application through the creation of change requests which are classified as bugs, feature requests, or tasks. | 2014-131 | |
| ZenDesk | ZenDesk included data related to user requests for assistance in using the OpenCCC applications. | Undetermined | ZenDesk is used to intake requests for end user assistance and to document interactions between the Help Center staff and the end user as the request is resolved in the Community Colleges’ OpenCCC application. | 2014-131 | |
| California Correctional Health Care Services | Contract Medical Database (CMD) | CMD contained more than 9.5 million records of contracted health care services. | Undetermined | The Contract Medical Database is a repository for data elements on claims submitted for contract health care services. | 2013-120 |
| California Department of Education | California Longitudinal Pupil Achievement Data System (CALPADS) | CALPADS contained nearly 128,000 school level enrollment records for the academic year 2011–12. | Undetermined | CALPADS was created to enable California to meet federal requirements delineated in the No Child Left Behind Act of 2001, which increases accountability for student achievement. | 2014-130 |
| Armed Prohibited Persons System (APPS) | As of April 2015, the APPS database contained more than 257,000 historical records for persons with more than 756,000 associated weapons. | Undetermined | The APPS database houses information on persons who purchased or acquired a handgun(s) on or after January 1, 1996, or registered an assault weapon(s), and subsequently became prohibited from owning and/or possessing firearms under state or federal law. | 2015-504 | |
| Mental Health Firearms Prohibition System (mental health database) | As of December 2014, the mental health database contained information regarding 172 active and reportable mental health facilities and 361 courthouses. | Undetermined | The mental health database is an inquiry‑only database containing firearms eligibility information on persons prohibited from owning or possessing firearms due to a mental health disorder per sections 8100 and 8103 of the California Welfare and Institutions Code. | 2015-504 | |
| Application Lifecycle Management | The database contained nearly 19,000 records of defects. | Undetermined | To track the life cycle of defects from identification to resolution in support of the information technology services provided by the California Healthcare Eligibility, Enrollment, and Retention System (CalHEERS) project. | 2014-131 | |
| Remedy IT Service Management Suite | The database contained nearly 164,000 records of incidents and problems. | Undetermined | To track the life cycle of the Help Desk tickets from identification to resolution in support of the information technology services provided by the CalHEERS project. | 2014-131 | |
| California Health Facilities Financing Authority | Master grant and disbursement spreadsheets for the Children’s Hospital Bond Act of 2004 and 2008 | The Children’s Hospital Bond Act of 2004 and 2008 spreadsheets contain nearly 70 and 60 disbursements, respectively, as of February 2015. | Yes | To identify the grant awards and disbursements that the authority has made as of February 28, 2015. | 2015-042 |
| Balancing Accounts Spreadsheets | The spreadsheets contained balancing accounts for 24 energy, natural gas, and water utilities each having from one to 52 balancing accounts. | Undetermined | To identify the balancing accounts that the CPUC directed utilities to establish and maintain. | 2013-109 | |
| CPUC’s data as maintained in Finance’s CALSTARS | The CALSTARS data for the Transportation Reimbursement Account contained more than $14 million in revenue and nearly $11 million in expenditures for fiscal year 2012–13. | Undetermined | An automated organization and program cost‑accounting system to accurately and systematically account for all revenue, expenditures, receipts, disbursements, and property of the State. | 2013-130 | |
| Case Tracker | Case Tracker contained nearly 1,200 closed investigations for fiscal years 2009–10 through 2012–13. | Undetermined | To store investigative information for the CPUC’s Transportation Enforcement Branch. | 2013-130 | |
| Transportation Management Information System | The Transportation Management Information System recorded more than $6.6 million in fees collected from Charter‑Party Carriers and Passenger Stage Corporations for fiscal year 2012–13. | Undetermined | To track fees paid, operation authorities, license, and other information about vehicles regulated by the CPUC. | 2013-130 | |
| Work Tracking System | The Work Tracking System recorded nearly 330,000 hours for the Transportation Enforcement Branch for fiscal years 2009–10 through 2012–13. | Undetermined | To track timesheets for employees of the CPUC. | 2013-130 | |
| Consumer Information Management System (CIMS) | For the period of July 1, 2011, through June 30, 2014, CIMS contains more than 152,000 cases received by the Consumer Affairs Branch at the CPUC. | No# | CIMS is used to better process, track, and analyze consumer complaints. | 2014-120 | |
| Oracle Financial System | For the period July 2010 through March 2014, the Oracle Financial System contained information pertaining to Judicial Council and Judicial Branch Facility Program expenditures of nearly $1.15 billion. | No# | The Judicial Council uses the Oracle Financial System to issue purchase orders and record certain procurement activity. | 2014-107, 2015-302 | |
| Judicial Council’s data as maintained in the State Controller’s Uniform State Payroll System | The Uniform State Payroll System contained nearly 47,000 payroll records for Judicial Council employees for the period of July 1, 2010, through March 31, 2014. | Undetermined | The State Controller uses the Uniform State Payroll System to process the State’s payroll and personnel transaction documents. | 2014-107 | |
| Phoenix Financial System | For the period July 1, 2014, through December 31, 2014, the Phoenix Financial System contained information pertaining to 5,627 contracts that were originally valued at over $153 million for the superior courts. | No# | The Judicial Council uses information from the Phoenix Financial System, in part, to compile the semiannual reports it submits to the Legislature and California State Auditor. The Phoenix Financial System contains procurement and payment information related to the superior courts. | 2015-302 | |
| State Bar of California | Discipline Case Tracking System (discipline database) | As of February 2015, the State Bar’s discipline database contained more than 240,000 discipline and regulatory cases, more than 94,000 reportable action items, and more than 442,000 inquiries. | Yes, Undeterminedll | To track complaints against attorneys from receipt, through investigation and litigation, to final disposition. | 2015-030 |
| California Leave Accounting System (CLAS) | CLAS contained more than 37 million records containing month end summary leave information. | No | To perform a variety of functions necessary to accurately track leave system eligibility, state service credits, and leave benefit activity. | 2012-603 | |
| Employment History Database | The Employment History Database contained more than 19 million month‑end records containing employee salary information. | Undetermined | Electronic database containing personnel records for state employees. | 2012-603 | |
| Uniform State Payroll System | The Uniform State Payroll System included more than 50 million payroll records. | Yes | The State Controller uses the Uniform State Payroll System to process the State’s payroll and personnel transaction documents. | 2013-111 | |
| Corporate Data Warehouse (CDW) | The CDW included information on up to 56 UC Los Angeles Medical Center employees and 129 UC San Francisco Medical Center employees who received annual compensation in excess of $200,000 each from 2009 through 2012. | Undetermined | The CDW stores corporate systems data (both current and historical) from the Corporate Personnel System, which provides Office of the President management and staff with demographic, personnel, and pay activity data on employees paid at the ten UC campuses, the Office of the President, the Division of Agricultural and Natural Resources, the Lawrence Berkeley National Laboratory, Hastings College of Law, and the Associated Students of UC Los Angeles. | 2013-111 | |
| Decision Support System | The Decision Support System included information on up to 56 UC Los Angeles Medical Center employees and 129 UC San Francisco Medical Center employees who received annual compensation in excess of $200,000 each from 2009 through 2012. | Undetermined | Provides an integrated, universal source for reporting and business intelligence decision making across the UC system, including the Lawrence Berkeley National Laboratory, campus medical centers, and other locations. Includes payroll and personnel data used for analysis, planning, and forecasting. | 2013-111 | |
| Davis Financial Information System (DaFIS) | DaFIS recorded nearly $557,000 in sales revenue for fiscal year 2010–11. | Undetermined | Prior to being retired in January 2016, DaFIS was the financial system used previous to the Kuali Financial System, which performed a variety of transactions related to accounts payable and receivable, purchasing, capital asset management, chart of accounts, and other financial activities for UC Davis. | 2014-121 | |
| UC Davis’ financial data as maintained in UC Los Angeles’ Financial System General Ledger Applications | The Financial System General Ledger Applications recorded nearly $63 million in patent royalties for fiscal year 2013–14. | Undetermined | The Financial System General Ledger Applications maintains the official financial book of record for UC Los Angeles, UC Office of the President, and UC Merced. | 2014-121 | |
| Kuali Financial System (KFS) | KFS contained more than $2.5 million expenses for the Strawberry Breeding Program for fiscal year 2013–14. | Undetermined | To perform a variety of transactions related to accounts payable and receivable, purchasing, capital asset management, chart of accounts, and other financial activities for UC Davis. | 2014-121 | |
| MyTravel System (MyTravel) | MyTravel recorded more than $24,000 in travel expenditures for the Strawberry Breeding Program for fiscal year 2013–14. | Undetermined | MyTravel is the online travel and entertainment expense reporting system for UC Davis. | 2014-121 | |
| UC Davis’ patent data as maintained in the UC Office of the President’s Patent Tracking System (PTS) | PTS recorded more than $65 million in patent royalties for fiscal year 2013–14. | Undetermined | PTS is a systemwide application for technology transfer activities such as licensing and financial information, invention disclosure, and patent prosecution. | 2014-121 | |
| Payroll and Personnel System (PPS) |
The PPS contains more than $1.3 million in total salaries for fiscal year 2013–14. | Undetermined | To ensure that all employees are paid properly and in a timely fashion and to support payroll‑related reporting requirements of both UC Davis and external agencies. | 2014-121 | |
* Some of the departments have changed their names subsequent to the issuance of our audits during 2014 and 2015. For these departments and purposes of this report, we refer to the department by its current name.
† Some departments may have replaced their information systems (system) subsequent to the issuance of our audits during 2014 and 2015.
‡ In those instances where the assessment is No or Undetermined, we recognize that the data limitations we identified may affect the precision of the numbers we presented in our reports. However, there was sufficient evidence in total to support our audit findings, conclusions and recommendations.
§ The reliability assessment relates to the purposes for which we tested the system’s data during the audit. The department’s use of the system’s data is usually, but not always, similar to our use of the system’s data.
ll A single system may have different assessments. For example, data that we used for one audit purpose was accurate and complete, whereas data from the same system used for a separate purpose was not.
# Our data reliability assessment, which relied upon a review of selected system controls, based the determination of not sufficiently reliable on Section 6.71b(1) of the U.S. Government Accountability Office’s December 2011 version of Government Auditing Standards, which states that evidence is not sufficient or not appropriate when using the evidence carries an unacceptably high risk that it could lead to an incorrect or improper conclusion.
Respectfully submitted,
ELAINE M. HOWLE, CPA
State Auditor
Footnotes
1The security standards consist of 64 different compliance sections. In addition, they identify the National Institute of Standards and Technology Special Publication 800‑53 and the Federal Information Processing Standards as the minimum information security control requirements that reporting entities must meet when planning, developing, implementing, and maintaining their information system security controls. The security standards also reference the Statewide Information Management Manual, which contains additional standards and procedures that address more specific requirements or needs that are unique to California. Go back to text.