Following the discovery of apparent anomalous data reported to the Monitoring Unit for the Education Image Gallery (EIG) service in 2009/10 Q3 EDINA were asked to comment on the anomalies and provide revised data in the event that inaccuracies were present. EDINA's investigations discovered that session data for the service had, in fact, been significantly under-reported since December 2008.
EDINA has explained that the error was caused by changes made to the service during the switch from Athens authentication to the use of the UK federation.
While revised session data have been submitted the corresponding search data (indicative of actual use of the service) was unaffected.
Table 1 shows the extent of the anomaly in the monthly collated session data submitted to the Monitoring Unit.
| Sessions | |||
|---|---|---|---|
| Date | Original | Revised | Difference |
| Dec-08 | 2,511 | 2,586 | 3% |
| Jan-09 | 1,653 | 1,706 | 3% |
| Feb-09 | 1,242 | 1,815 | 46% |
| Mar-09 | 1,135 | 1,799 | 59% |
| Apr-09 | 640 | 1,011 | 58% |
| May-09 | 697 | 1,083 | 55% |
| Jun-09 | 616 | 954 | 55% |
| Jul-09 | 526 | 766 | 46% |
| Aug-09 | 523 | 732 | 40% |
| Sep-09 | 1,258 | 1,766 | 40% |
| Oct-09 | 1,590 | 2,445 | 54% |
| Nov-09 | 1,319 | 2,179 | 65% |
| Dec-09 | 706 | 1,172 | 66% |
| Jan-10 | 839 | 1,410 | 68% |
| Feb-10 | 974 | 1,630 | 67% |
| Mar-10 | 1,165 | 2,296 | 97% |
| Apr-10 | 181 | 1,098 | 506% |
| Total | 17,575 | 26,448 | +8,873 50% |
A number of Monitoring Unit reports are affected by the resubmitted data: Quarterly Reports, Services Portfolio Review and 2010 Funding Body analysis.
Quarterly reports published by the Monitoring Unit since 2008/09 Q2 have made reference to EIG session data and in particular the 2009/10 Q1 & Q2 reports included graphs showing trends in the use of this service.
Figure 1a shows the effects on the long-term usage trend of the resubmitted data; Figure 1b highlights the differences on just the period under consideration.
Figure 1a: EIG sessions reported August 2007 to April 2010
Figure 1b: EIG sessions reported December 2008 to April 2010, showing extent of data resubmissionThe 2009 Services Portfolio Review paperwork supplied by the Monitoring Unit did not include usage data from AY2008/09; the effects of the erroneous data would not have been apparent.
The 2010 Services Portfolio Review (SPR) information sheets prepared by the Monitoring Unit included usage trend data and projections of likely future usage to August 2012.
Figure 2a is a copy of the graph included in the paperwork supplied to the SPR activity; Figure 2b shows the effects on projected activity based on the resubmitted data.
Figure 2a: EIG 2010 Services Portfolio Review sessions projection
Figure 2b: EIG sessions projection based on resubmitted dataUse of the WebFeat federated search software at the University of Edinburgh between October 2007 and December 2008 created artefacts in the underlying session usage data. Figure 3a shows a copy of the Figure 2a data with sessions from the University of Edinburgh removed. Figure 3b shows the effects of the resubmitted data on projected activity.
Figure 3a: EIG 2010 Services Portfolio Review sessions projection (excl. WebFeat)
Figure 3b: EIG sessions projection based on resubmitted data (excl. WebFeat)The effects of the resubmitted data to this latter model are significant and show a steady-state or even possible future increase in service usage.
Analysis completed earlier in the year examining use by the individual sectors of the four UK nations has been updated. The effects of the resubmitted data on this analysis are significant.
A revised copy of the analysis sheet is attached and a copy of the original sheet is available for comparison.
Downloads:
Last updated by Ian Cooper on Friday, August 13, 2010.
