Report 2016-129 Recommendation 6 Responses

Report 2016-129: K-12 High Speed Network: Improved Budgeting, Greater Transparency, and Increased Oversight Are Needed to Ensure That the Network Is Providing Reliable Services at the Lowest Cost to the State (Release Date: May 2017)

Recommendation #6 To: Imperial County Office of Education

To better guarantee that network upgrades are necessary and are achieved at the lowest possible cost to the State, ICOE should develop a formal methodology for reviewing circuit capacity needs. This methodology should include consideration of multiyear trends in network traffic and the implications prospective upgrades may have for other parts of the network. Doing so would not only assist ICOE when determining the magnitude of circuit upgrades relative to the cost involved, but would also help ICOE determine whether it can delay upgrades until it can establish those upgrades' eligibility for E-Rate subsidies.

Annual Follow-Up Agency Response From October 2018

Over the course of the past year the program has implemented a formal methodology which includes extending data retention on utilization for a minimum of five years, soliciting additional inputs from node sites and district technologists and preparing trending information and over-subscription rates on all node sites under consideration for upgrade. During the bidding process, bids are solicited for multiple bandwidths to ensure as many options as possible are considered in selection of appropriate bandwidths for each node site. The upgrade decision process has been memorialized and is available for review on the K12HSN website at www.k12hsn.org under "Policies" and is being provided via email to Mark Reinardy.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Based on the range of factors ICOE must consider when making upgrade decisions, we stated in our assessment of its one-year response that we would be looking for evidence that ICOE has consistently and systematically considered all relevant measures when evaluating implementation of this recommendation. In addition to the written process for considering upgrade decisions that ICOE describes in its current response, ICOE provided evidence that it collected and considered factors such as cost, multi-year network usage trends, and "downstream" capacity and usage by offices of education and school districts. Using this information, ICOE made preliminary upgrade determinations that it provided to the Department of Education and Department of Finance in October 2018 for their review, along with its proposed fiscal year 2019-20 K12HSN budget. Based on the information ICOE has collected and the evidence it presented to us, which indicates it has performed equivalent analyses for a range of network upgrades, we conclude that ICOE has fully implemented our recommendation. However, we intend to follow up with ICOE after the fiscal year 2019-20 upgrade decisions have been finalized in order to thoroughly validate ICOE's application of its methodology and ensure its analyses are based on complete and accurate information.

1-Year Agency Response

ICOE has developed a new template to document factors considered for the upgrade recommendations to Education. This template was created in accordance with Ed. Code section 11800 parameters and aims to document and memorialize all factors, qualitative and quantitative, considered for upgrade decisions. These factors will continue to drive the decision-making process for K12HSN administration, in addition to ever-evolving market conditions, overall network operation and long-term strategic planning. The methodology includes:

New timeline to align with state budget planning cycles as presented in Recommendation #3

Oversubscription rates downstream (DataLINK) for each node and a comparison to statewide averages

Other downstream inputs from DataLINK including district-level information on number of devices supported, cloud-based activities and other triggers for utilization growth

Trending data from previous years (Data collection on node site traffic usage in the months of May and October which represent peak school year traffic).

Network reliability data based on the CENIC Network Operation Center announcements

Multiple bandwidth increase options and financial impact analysis based on RFP responses received

Review and receive input from the Network Implementation Committee, Regional leads, CENIC Engineers and K12HSN staff

Based on 2018-19 decisions being due in March 2018 to meet E-rate deadlines, the full implementation of the new processes will begin in the 2018 calendar year in preparation for the 2019-2020 budget cycle.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Based on its 1-year response and the supporting documentation ICOE provided with respect to this recommendation's status, we determined that the recommendation is partially implemented insofar as ICOE has created and provided general written processes and timelines by which it will utilize the recommended methodology. In addition to these documents, ICOE provided an evaluation template that reflects some, but not all, of the measures we identified during our audit as important considerations for upgrade decisions.

When evaluating ICOE's implementation of this methodology as part of the 2019-2020 budget cycle referenced in its response, we will be looking for evidence that ICOE has consistently and systematically considered all relevant measures as part of selecting network upgrades.

6-Month Agency Response

ICOE convened the Network Implementation Committee (NIC) on September 7, 2017 to discuss the proposed new process for network upgrades in line with other recommendations of the audit. The methodology includes:

New timeline to align with state budget planning cycles

Improved tool for collecting downstream capacity information from school districts (DataLINK).

Develop preliminary list of node upgrades based on:

Oversubscription rates downstream (DataLINK)

Trending data (Data collection on node site traffic usage in the months of May and October (peak school year traffic).

Bandwidth usage data from Orion and Cricket monitoring tools

Technical review process by CENIC engineers

Review process by the NIC on proposed upgrades

The new process is scheduled to be implemented starting the 2018 calendar year in preparation for the 2019-2020 budget cycle.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

As indicated by its response, ICOE has identified a range of factors it plans to consider as components of its methodology for identifying the need and magnitude of network capacity upgrades. However, ICOE has not yet formally applied those components in the context of the network planning process in order to reach upgrade decisions. ICOE intends to do so as part of its efforts leading up to the submission of its fiscal year 2019-2020 K12HSN budget. Our expectation regarding that process, which aligns with changes made to the state Education Code regarding the administration of K12HSN, is that through the process, ICOE will produce a record of which factors ultimately drove ICOE's determinations for the specific network upgrades in question. Using information from that process, we will be able to evaluate the extent to which ICOE has implemented its methodology.

60-Day Agency Response

ICOE adjusted the settings on the Orion/Solarwinds monitoring system by June 15, 2017 so that average utilization per day at each node site is being stored for five years (1,825 days). The multi-year data reflecting network traffic will be used in processing decisions to maintain current bandwidth or increase bandwidth to meet demonstrated need. This process will be informed by a re-energized effort to collect full information from node site liaisons on the anticipated changes in downstream demand, school district initiatives and all other relevant factors. The implications that each prospective upgrade will have on other parts of the network will also be considered.

As mentioned before the Network Implementation Committee (NIC) will be called upon as an advisory panel to guide the effort to formally collect information on current utilization and anticipated growth over the next 6, 12 and 18 months. The support anticipated from node site liaisons will be communicated via the NIC, the annual node site meeting staged during an annual conference for technologists, and via email announcements to the technical and administrative contacts lists maintained by ICOE. The K12HSN will work with node site liaisons to conduct data collection reflecting the plans of the districts in their service areas and, where available, will collect links to any URLs available for monitoring usage. The whole effort will be documented into a written methodology that will be publicly available.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2016-129

Agency responses received are posted verbatim.