Report 2010-102 Recommendations
When an audit is completed and a report is issued, auditees must provide the State Auditor with information regarding their progress in implementing recommendations from our reports at three intervals from the release of the report: 60 days, six months, and one year. Additionally, Senate Bill 1452 (Chapter 452, Statutes of 2006), requires auditees who have not implemented recommendations after one year, to report to us and to the Legislature why they have not implemented them or to state when they intend to implement them. Below, is a listing of each recommendation the State Auditor made in the report referenced and a link to the most recent response from the auditee addressing their progress in implementing the recommendation and the State Auditor's assessment of auditee's response based on our review of the supporting documentation.
Recommendations in Report 2010-102: Administrative Office of the Courts: The Statewide Case Management Project Faces Significant Challenges Due to Poor Project Management (Release Date: February 2011)
Recommendations to Administrative Office of the Courts | ||
---|---|---|
Number | Recommendation | Status |
1 | To better manage costs of future IT projects, the AOC should estimate costs at the inception of projects. |
Fully Implemented |
2 | To better manage costs of future IT projects, the AOC should employ appropriate budget and cost management tools to allow it to appropriately budget, track, manage, and estimate costs. |
Fully Implemented |
3 | To better manage costs of future IT projects, the AOC should ensure that cost estimates are accurate and include all relevant costs, including costs that superior courts will incur. |
Fully Implemented |
4 | To better manage costs of future IT projects, the AOC should disclose costs that other entities will likely incur to the extent it can reasonably do so. |
Fully Implemented |
5 | To better manage costs of future IT projects, the AOC should update cost estimates on a regular basis and when significant assumptions change. |
Fully Implemented |
6 | To better manage costs of future IT projects, the AOC should disclose full and accurate cost estimates to the Judicial Council, the Legislature, and stakeholders from the beginning of projects. |
Fully Implemented |
7 | To better manage costs of future IT projects, the AOC should ensure that it has a long-term funding strategy in place before investing significant resources in a project. |
Fully Implemented |
8 | Although the Judicial Council has the legal authority to compel the courts to adopt CCMS, to better foster superior court receptiveness to deploying CCMS, the AOC should continue to work with the superior courts that have deployed the civil system to ensure it is addressing their concerns in a timely and appropriate manner. |
Fully Implemented |
9 | Although the Judicial Council has the legal authority to compel the courts to adopt CCMS, to better foster superior court receptiveness to deploying CCMS, the AOC should work with superior courts to address concerns about hosting data at the California Court Technology Center (Technology Center). Further, the AOC should take steps to ensure that superior courts do not lose productivity or efficiencies by hosting data at the Technology Center. |
Fully Implemented |
10 | To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should obtain IV&V and IPO services at the beginning of the projects and ensure this independent oversight is in place throughout and follows best practices and industry standards appropriate for the size and complexity of the project. |
Fully Implemented |
11 | To understand whether CCMS is a cost-beneficial solution to the superior courts' case management needs, the AOC should continue with its planned cost-benefit study and ensure it completes this study before spending additional significant resources on the project. The AOC should ensure that this study includes a thorough analysis of the cost and benefits of the statewide case management project, including a consideration of costs and benefits it believes cannot be reasonably quantified. The AOC should carefully evaluate the results of the study and present a recommendation to the Judicial Council regarding the course of action that should be taken with CCMS. Further, the AOC should fully share the results of the study as well as its recommendation to all interested parties, such as the superior courts, justice partners, the Legislature, and the Information Office. The AOC should update this cost-benefit analysis periodically and as significant assumptions change. |
Will Not Implement |
12 | To ensure the statewide case management project is transparent, the AOC should make sure all key decisions for future activities on CCMS are documented and retained. |
Fully Implemented |
13 | To ensure its contract with the development vendor protects the financial interests of the State and the judicial branch, the AOC should consider restructuring its current contract to ensure the warranty for CCMS is adequate and covers a time period necessary to ensure that deployment of CCMS has occurred at the three early adopter courts and they are able to operate the system in a live operational environment. |
Will Not Implement |
14 | If the Judicial Council determines that CCMS is in the best interest of the judicial branch and it directs the AOC to deploy the system statewide, assuming funding is available, the AOC should ensure that any contract it enters into with a deployment vendor includes cost estimates that are based on courts’ existing IT environments and available resources to assist with deployment activities. |
Will Not Implement |
15 | If the Judicial Council determines that CCMS is in the best interest of the judicial branch and it directs the AOC to deploy the system statewide, assuming funding is available, the AOC should ensure that any contract it enters into with a deployment vendor includes well-defined deliverables. |
Will Not Implement |
16 | If the Judicial Council determines that CCMS is in the best interest of the judicial branch and it directs the AOC to deploy the system statewide, assuming funding is available, the AOC should ensure that any contract it enters into with a deployment vendor includes adequate responsibility is placed on the vendor for conducting key steps in the deployment of the system. |
Will Not Implement |
17 | The Judicial Council should make certain that the governance model for CCMS ensures that approval of contracts and contract amendments that are significant in terms of cost, time extension, and/or change in scope occur at the highest and most appropriate levels, and that when contracts or contract amendments above these thresholds are approved, that the decision makers are fully informed regarding both the costs and benefits. |
Will Not Implement |
18 | To ensure that any future IT projects are in the best interest of the judicial branch and the State, the AOC should complete a thorough analysis of the project’s cost and benefits before investing any significant resources and time into its development, and update this analysis periodically and as significant assumptions change. |
Fully Implemented |
19 | To ensure that any future IT projects are in the best interest of the judicial branch and the State, the AOC should document and retain all key decisions that impact the project in general, including the goals of the project. |
Fully Implemented |
20 | To ensure that any future IT projects are in the best interest of the judicial branch and the State, the AOC should better structure contracts with development and deployment vendors to protect the financial interests of the judicial branch and ensure the contracts provide for adequate warranty periods. |
Fully Implemented |
21 | To ensure that the financial implications of the statewide case management project are fully understood, the AOC should report to the Judicial Council, the Legislature, and stakeholders a complete accounting of the costs for the interim systems and CCMS. This figure should be clear about the uncertainty surrounding some costs, such as those that the AOC and superior courts will incur for deployment of CCMS. |
Will Not Implement |
22 | The AOC should require superior courts to identify their past and future costs related to the project, particularly the likely significant costs that superior courts will incur during CCMS deployment, and include these costs in the total cost. |
Will Not Implement |
23 | The AOC should be clear about the nature of the costs that other entities, such as justice partners, will incur that are not included in its total. |
Will Not Implement |
24 | The AOC should update its cost estimate for CCMS on a regular basis as well as when significant assumptions change. |
Will Not Implement |
25 | To address the funding uncertainty facing CCMS, the AOC should work with the Judicial Council, the Legislature, and the governor to develop an overall strategy that is realistic given the current fiscal crisis facing the State. |
Will Not Implement |
26 | Although the Judicial Council has the legal authority to compel the courts to adopt CCMS, to better foster superior court receptiveness to deploying CCMS, the AOC should use the results from its consultant’s survey of the superior courts to identify and better understand the courts’ input and concerns regarding CCMS, including the manner in which the project has been managed by the AOC. To the extent the survey results indicate courts have significant concerns regarding CCMS or that they believe their case management systems will serve them for the foreseeable future, the AOC should take steps to address these concerns and overcome any negative perceptions and modify its deployment plan for CCMS accordingly. |
Will Not Implement |
27 | The AOC should continue working with local and state justice partners to assist them in their future efforts to integrate with CCMS, and in particular provide local justice partners the information needed to estimate the costs involved. |
Will Not Implement |
28 | Before embarking on future statewide IT initiatives and to ensure it secures appropriate support from users of the systems being proposed, the AOC should determine the extent to which the need for the IT initiative exists, including the necessary information to clearly demonstrate the extent of the problem the IT initiative will address. |
Fully Implemented |
29 | Before embarking on future statewide IT initiatives and to ensure it secures appropriate support from users of the systems being proposed, the AOC should take steps to ensure that superior courts support the solution the AOC is proposing to address the need, which could include conducting a survey of courts to determine their level of support. |
Fully Implemented |
30 | Before embarking on future statewide IT initiatives and to ensure it secures appropriate support from users of the systems being proposed, the AOC should if necessary, determine whether other stakeholders, including local and state justice partners, support the IT initiative. |
Fully Implemented |
31 | To provide for an appropriate level of independent oversight on CCMS, the AOC should expand and clarify the scope of oversight services and require that oversight consultants perform oversight that is consistent with best practices and industry standards. |
Will Not Implement |
32 | To ensure that no gaps in oversight occur between CCMS development and deployment, the AOC should ensure that it has IV&V and IPO services in place for the deployment phase of CCMS. Further, to allow for independent oversight of the IV&V consultant, the AOC should use separate consultants to provide IV&V and IPO services. |
Will Not Implement |
33 | To ensure no significant quality issues or problems exist within CCMS, the AOC should retain an independent consultant to review the system before deploying it to the three early adopter courts. This review should analyze a representative sample of the requirements, code, designs, test cases, system documentation, requirements traceability, and test results to determine the extent of any quality issues or variances from industry standard practices that would negatively affect the cost and effort required of the AOC to operate and maintain CCMS. If any quality issues and problems identified by this review can be adequately addressed, and system development can be completed without significant investment beyond the funds currently committed, the AOC should deploy it at the early adopter courts during the vendor’s warranty period. |
Will Not Implement |
34 | To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should employ separate firms for IV&V and IPO services to allow for the IPO consultant to provide independent oversight on the IV&V consultant as well as the project team’s response to IV&V findings. |
Fully Implemented |
35 | To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should ensure that the staff performing IV&V and IPO services have experience and expertise that is commensurate with the size, scope, and complexity of the project they are to oversee. |
Fully Implemented |
36 | To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should ensure that independent oversight is not restricted in any manner and that all parties—the IV&V and IPO consultants, senior management, the project management team, and the development vendor—understand that the IV&V and IPO consultants are to have complete access to all project materials. |
Fully Implemented |
37 | To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should address promptly and appropriately the concerns that independent oversight consultants raise. |
Fully Implemented |