Measuring Up: Assessing Accuracy of Reported Use and Impact of Digital Repositories

This work is part of the IMLS-funded grant "Measuring Up: Assessing Accuracy of Reported Use and Impact of Digital Repositories" (IMLS Award Number: LG-06-14-0090-14) which aims to better improve data collection and information sharing for institutional repositories and digitized collections. "Measuring Up" is led by Montana State University and includes partnerships with OCLC Research, the Association of Research Libraries and the University of New Mexico.

The goals of the grant project are to:

• Gather data about the state of web analytics reporting in academic libraries
• Establish best practices for web metrics for digital collections and IR
• Raise awareness of privacy issues associated with web analytics software and propose solutions
• Establish a common understanding of what will be measured and provide an assessment framework that identifies digital collection and IR metrics in the spirit of IMLS Strategic Goals
• Develop a structured data model that improves machine access and measurement of digital collections and IR content

Outputs

RAMP Prototype

The Repository Analytics & Metrics Portal (RAMP) is a prototype web service that improves the accuracy of institutional repository (IR) analytics and provides IR managers with the following capabilities:

1. Persistent access to accurate counts of file downloads from IR.
2. Implementation with minimal training or configuration.
3. The potential to aggregate IR metrics across organizations for consistent benchmarking and analysis

Learn More about RAMP

Publications

• OBrien, Patrick, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. 2016. “Undercounting File Downloads from Institutional Repositories.” Journal of Library Administration 56, no. 7: 854-874. http://dx.doi.org/10.1080/01930826.2016.1216224.
  
  
• OBrien, Patrick, Kenning Arlitsch, Jeff Mixter, Jonathan Wheeler, and Leila Sterman. 2017.“RAMP - The Repository Analytics and Metrics Portal: A Prototype Web Service that Accurately Counts Item Downloads from Institutional Repositories.” Library Hi Tech 35, no. 1: 144-158. http://dx.doi.org/10.1108/LHT-11-2016-0122.

Background

Measurement is important for libraries. Measuring collection size, usage statistics, patron visitation and cost of materials are a few of the metrics that libraries use to support budget proposals and show relevancy in the community. These measurements are generally transferred to digital collections as well, resulting in statistics that illustrate website access and download rates for digital collections and items. There are a lot of ways to collect metrics for digital collections and IR and this has resulted in a lack of consistency and standards. This proposal intends to examine challenges associated with collecting digital statistics as well as propose a set of standards and best practices for collecting and analyzing metrics.

Impact

This project will:

• Examine the challenges of collecting statistics for digital collections and IR
• Provide recommendations and best practices for collection and analyzing metrics for digital collections and IR

OCLC's role

• OCLC's primary contribution is a data model defined using linked data principles that makes it easier to collect web metrics for digital collections and institutional repositories.  Contact: Jeff Mixter, Software Engineer.
• OCLC Research staff are also participating in an advisory panel for the project. Contacts: Ricky Erway and Roy Tennant, Senior Program Officers; and Jean Godby, Senior Research Scientist.

Team

Jean Godby

Jeff Mixter