Duke University Library
Academic Library Case Study
Evidence-Based Management
Duke University Library
Yvonne Belanger, Head, Assessment and Planning
Duke University Challenge: Collecting and Managing Data from Multiple Libraries with Different Processes
Duke University is a private institution with several libraries in addition to the William R. Perkins Library, the main library that serves the whole campus. Over the past five years, the library has grappled with how to manage the complexity of metrics that need to be reported to various state and national organizations.
According to Yvonne Belanger, Head, Assessment and Planning, requests for data come from many places and each requesting body wants different statistics, requiring the library to count and slice data in multiple ways. Statistics have to be aggregated from different departments, with no one department responsible for collecting and managing the data. A transition of responsibility from the Collection Development Department to the Department of Assessment and Planning presented an opportunity to re-examine the process being used. It quickly became clear that the library wasn’t taking advantage of technology to improve data gathering.
“We wanted to move from a paper- and spreadsheet-based system and aggregate the data used for all these different surveys including ARL, ASERL, ACRLAssociation of College and Research Libraries, the federal government, and the state of North Carolina,” Belanger said. “Our previous process didn’t readily footnote how numbers were derived so as staff left we lost the institutional knowledge of what the numbers meant. Everyone had a different way of tracking and when there was a discrepancy from one year to the next we couldn’t understand the trend. We needed to reduce the amount of manual labor involved, and we needed a better centralized system. A web-based solution seemed logical for collecting and aggregating our statistics.”
The LibPASLibrary Performance Assessment Service Solution: Rapid Implementation of Web-based Statistics Management Tool
Belanger said the library wanted a technology solution that could adapt Duke’s existing processes and improve upon it. After talking with other libraries that had similar institutional complexities, they determined that LibPASLibrary Performance Assessment Service could deliver what they needed and provided a good value compared to what it would have taken to create this type of tool internally. “We saw that LibPASLibrary Performance Assessment Service would help improve the integrity and quality of our data and retain institutional knowledge about how we were capturing data and what the numbers meant.”
The library needed to move quickly through set up and implementation. To speed the process they talked to a peer library that had implemented LibPASLibrary Performance Assessment Service and had thought through many of the same questions Duke had. Belanger worked one on one with Counting Opinions’ staff to walk through the functionality of the system. “We were very clear on what we wanted to do. I worked with a group of data providers on campus to talk through the hierarchy of data collection before we moved forward. We needed to make decisions up front about which units would report separately and which would report together. It was very helpful to involve end-users of the system early in the process so they understood where they fit into the big picture before they started adding data. The one on one help from Counting Opinions helped us focus on what was important.”
To help Duke move quickly through implementation, Counting Opinions took specific typical data elements (e.g., gate count and checkouts) and used those to see how the system could be structured from the input side to the output side. They began setting up the system with these samples, gradually building it out with a very hands-on approach.
Belanger said they started setup in summer 2011 and were ready to gather ARL statistics in October, a very rapid implementation by any standard. Setting up Performance Indicators wasn’t particularly complex because LibPASLibrary Performance Assessment Service already had the data elements from the ACRLAssociation of College and Research Libraries and NCESNational Center for Education Statistics surveys. “At that time, we didn’t have to think about setting up new PIsPerformance Indicator(s), also Data Element(s) – we were copying already defined ones,” she said.
A critical part of the implementation was setting up virtual views for data providers so they could see only those fields they needed to fill in. Duke wanted these customized views for data entry to mirror what they had been doing previously. Before LibPASLibrary Performance Assessment Service, the library sent out customized questionnaires to each data provider, which was labor intensive. LibPASLibrary Performance Assessment Service virtual views provide this same level of customization and also enabled Duke to display prior year’s data to data providers so they could see what had been reported previously and identify (and footnote) significant variances at the time of reporting. Belanger says the benefits of using LibPASLibrary Performance Assessment Service include:
- Create data transparency in a central place. LibPASLibrary Performance Assessment Service includes their internal footnotes so from year to year all data providers have the same interpretation of the numbers.
- Facilitate the transition to the new ARL metrics. Library staff will be able to copy the new Performance Indicators and not have to re-create them. This will save time in making the transition.
- Ability for individual units to run local reports. Belanger says LibPASLibrary Performance Assessment Service works well as a data repository and now they need to train staff on how to get the data out as well as put it in. The ability to de-centralize the ability to run reports will reduce the burden of relying on a limited number of staff and put data in the hands of those who need and want it.
- Save staff time in submitting statistics. Belanger says the real time savings will come when staff can start submitting their ARL statistics as soon as the fiscal year ends. Without having to create customized questionnaires, she hopes they can expedite the process of ARL reporting with a minimal effort.
- Monitor trends across units. The project has provided the library with a more complete picture of what the different units are counting. Right now, Belanger says, “Now we have key indicators in a central location to see and aggregate trends.”
Looking Ahead
In the first year of using LibPASLibrary Performance Assessment Service Belanger says the goal was to adapt an existing process and streamline it through the use of technology. But she sees ways LibPASLibrary Performance Assessment Service may be used going forward, beyond simply meeting their standard reporting obligations.
For example, she says, they want to use LibPASLibrary Performance Assessment Service to capture locally useful metrics not required for external reporting. These might include library instruction statistics beyond headcounts and e-resources cost per use, both of which are captured now but are not stored in a central system. The library also reports subject-specific metrics to organizations such as CEAL, the Council on East Asian Libraries. They aren’t currently using LibPASLibrary Performance Assessment Service for this, but they could. In general, Belanger says, she envisions multiple ways LibPASLibrary Performance Assessment Service might help them streamline other small-scale reporting obligations. The library also plans to evaluate whether LibSatLibrary Customer Satisfaction Management Service might enhance their current toolset for conducting user surveys.
Belanger says the library hopes to see a return on their investment in terms of the time they will save. In addition to their goal of reducing staff effort, they were also interested in improving the quality of the data-gathering process. “We have been able to engage more people directly in gathering statistics. Previously we tried to limit this because of the work involved. Now we are able to welcome the participation of more people and are creating a culture of assessment in the university. You build a culture of assessment when people understand their role and use data. This tool – LibPASLibrary Performance Assessment Service – facilitates that.”
About Duke University
Home of the Blue Devils, Duke University has about 13,000 undergraduate and graduate students and a world-class faculty helping to expand the frontiers of knowledge. The university has a strong commitment to applying knowledge in service to society, both near its North Carolina campus and around the world.
The university is served by a network of ten libraries. The William R. Perkins Library, Bostock Library, and David M. Rubenstein Rare Book & Manuscript Library comprise the university’s main library complex, which is joined on East Campus by the Lilly and Music Libraries, and by the Pearse Memorial Library at the Duke Marine Lab. Together with the separately administered libraries serving the schools of Business, Divinity, Law, and Medicine, they comprise one of the nation’s top 10 private research library systems. More information: http://library.duke.edu
About Counting Opinions
Facing constant competitive challenges, libraries and library organizations need better tools to understand and manage customer needs and to compete more effectively for scarce resources. In business since 2004, Counting Opinions provides libraries and library organizations with a cost-effective, evidence-based management solutions’ platform for the comprehensive management of their performance and customer satisfaction data. Solutions for custom surveys, open-ended customer feedback, trends, benchmarking, outcomes and peer comparisons are also available.
Note: Interview conducted and case study prepared by JAM Marketing LLC.
Please cite this article as: Duke University Library; Yvonne Belanger, Head, Assessment and Planning