Rutgers University Library
Academic Library Case Study
Evidence-Based Management
Rutgers University Library
Jeanne Boyle, Associate University Librarian for Planning and Organizational Research
The Rutgers Goal: Continuous Improvement in Customer Satisfaction and Library Performance
With 28 libraries, centers, and reading rooms on campuses in New Brunswick, Newark, and Camden, Rutgers University Libraries ranks among the nation’s top research library systems. The library wanted to better understand the needs of its diverse population and to have a more systematic method of measuring library performance. LibSatLibrary Customer Satisfaction Management Service (customer satisfaction management module) and LibPASLibrary Performance Assessment Service (Library Performance Assessment System) from Counting Opinions provided the solutions.
The Challenge: Understanding Customer Needs
In 2005 the library implemented a complex survey to help inform the development of a strategic plan. Although valuable information was gleaned, the survey represented a snapshot in time and high-level findings weren’t easily actionable. A close examination of the survey results revealed that the range of respondents didn’t match the university profile, undermining the statistical validity of the survey. In addition, given the diversity of the library system, it wasn’t feasible to do this type of survey annually.
In looking for a solution, library faculty and staff wanted a tool to continuously assess the needs of users in a way that would provide actionable information and demonstrate responsiveness to the university administration. According to Jeanne Boyle, Associate University Librarian for Planning and Organizational Research, “It’s important to show why the library is valuable. The accreditation department wants to see that we are doing assessment and we need to be sure we are providing our users with want they want. The goal, over time, is to show improvement.”
The LibSatLibrary Customer Satisfaction Management Service Solution: From Big Picture to Everyday Issues
Boyle says she was attracted to Counting Opinions and LibSatLibrary Customer Satisfaction Management Service because data gathering would be continuous and the library would have control over how often to implement the survey. The survey would provide information at a high level as well as at the practical, everyday issues level. “Understanding what users want is an ongoing challenge. With LibSatLibrary Customer Satisfaction Management Service we can produce reports by discipline to show how users prefer to get information. This is an extremely valuable tool for improving customer satisfaction.”
Rutgers implemented LibSatLibrary Customer Satisfaction Management Service with an eye-catching corner curl widget on thirteen web pages, with a pop-up graphic on the same pages to do random sampling. The graphics are part of the support Counting Opinions provides to help libraries market the survey. Following a mini-campaign in the fall of 2009, the library found Counting Opinions’ prediction on survey respondents to hold true – about one-third of the users answered the indepth 15-minute survey with the other 70% selecting the shorter, seven-minute survey. During the school year, the library averaged about 200 completed surveys a month, with a good cross-section from different disciplines.
“We are pleased with the results,” says Boyle. “The survey is very flexible so people can skip questions. We find out what our users want to tell us and we can respond in a timely way.” For example, LibSatLibrary Customer Satisfaction Management Service confirmed the need for a redesign of the library’s website and showed that users want more policy enforcement in areas such as noise control and computer use. The survey also demonstrated that the document delivery/interlibrary loan service area is highly valued but reference service didn’t achieve a comparable level of user satisfaction.
According to Boyle, LibSatLibrary Customer Satisfaction Management Service provides the data to correct problems and make informed management decisions. “We can work in a non-judgmental way with our reference staff because we have user data on service quality. Administrators can demonstrate why they need more comfortable seating or longer library hours and facilities managers have data to support building improvement requests.”
Boyle points to the following benefits of using LibSatLibrary Customer Satisfaction Management Service:
- Continuous feedback, not a snapshot in time, with actionable information. LibSatLibrary Customer Satisfaction Management Service is available on the web pages the library chooses for as long as they choose. Administrators can review positive and negative comments in the summary reports and set goals to address issues that have been raised.
- Ability to gather immediate feedback. LibSatLibrary Customer Satisfaction Management Service provides timely collecting and accessing of data to assess the impact of change, as, for example, when new software is installed.
- Data can be used to gain resources to support new initiatives. Increasing library hours, providing more printers, and better climate control in the buildings all require funding. Budget requests are more compelling when backed by LibSatLibrary Customer Satisfaction Management Service data, says Boyle.
- Ability to target different segments of the university population. The library can choose which web pages to display the survey on and can change the pages to ensure a cross-section of the university population is represented in the responses. Although the opt-in nature of the survey means it’s not statistically valid, this flexibility does result in valuable information being gathered from the entire university community.
- Ability to track trends over time. Rutgers is planning a new learning space at one of the libraries and will use LibSatLibrary Customer Satisfaction Management Service to measure customer satisfaction before the opening and then to track satisfaction trends on an ongoing basis.
- Ability to benchmark against other institutions. Although LibSatLibrary Customer Satisfaction Management Service is highly customizable, Boyle says the use of a standard survey will allow performance comparisons with peer libraries.
- Ease of use. Boyle says the Counting Opinions staff are very responsive and provide excellent support. LibSatLibrary Customer Satisfaction Management Service reports are easy to understand and share with managers and other staff. You don’t need a survey specialist on staff to make good use of LibSatLibrary Customer Satisfaction Management Service, she says.
Rutgers has collected enough LibSatLibrary Customer Satisfaction Management Service data to show year-to-year comparisons. Research staff produce library-by-library reports and distribute them for action and annual goal setting. “LibSatLibrary Customer Satisfaction Management Service is a tool that lets us communicate across a variety of management levels. We use it to demonstrate value and relevance to our administration. But we also use it to show our librarians how users prefer to find information. Over time we want to show improvement in the level of satisfaction expressed by our customers and LibSatLibrary Customer Satisfaction Management Service will help us do that.”
LibPASLibrary Performance Assessment Service Helps Tame the Library Performance Data Monster
Rutgers is in the process of implementing the second component of Counting Opinions’ solutions – the LibPASLibrary Performance Assessment Service performance assessment tool. The goal is to streamline the task of reporting annual statistics to ARL, ACRLAssociation of College and Research Libraries and NCESNational Center for Education Statistics as well as produce better reports for internal budgeting.
According to Boyle, the library has been gathering and managing data through an enormous Excel spreadsheet. One person was responsible for getting the data from each of the libraries, compiling it, and distributing it in printed form. Some areas were not always accurate, and producing the reports was burdensome.
Implementing LibPASLibrary Performance Assessment Service has provided the opportunity and the motivation to review what they were collecting and why. The goals were to improve accuracy, establish common definitions, and provide a central database for data input and extraction. “This year we want to produce our reports for ARL and NCESNational Center for Education Statistics on the system,” says Boyle. “No more huge spreadsheet on anyone’s desktop and all the middle managers who need data can generate their own reports. It’s also a huge benefit that LibPASLibrary Performance Assessment Service is a hosted system that does not need to be managed by our systems people.”
What to Collect, Why and How
The library established a lightning task force of three people to examine what data was being collected, to recommend a better way to collect it, and to identify data to stop collecting. Boyle says they expect LibPASLibrary Performance Assessment Service to help them collect data in different ways that will reduce the amount of time staff spend on collecting statistics. For example, they anticipate being able to import data directly from their integrated library system into LibPASLibrary Performance Assessment Service, eliminating the need for manual data entry.
Boyle says they contracted with Counting Opinions staff to import historical data, establish performance indicators, and set up user accounts. “Working in partnership with Counting Opinions has been critical to our implementation. Their staff understand what’s needed for ARL and NCESNational Center for Education Statistics and made it easy for us to take over and move forward.” The library is focusing initially on data that has to be collected for standard reports but they plan to move beyond that to enrich their data in new ways, including more granular instruction data than is required by ARL such as number of classes being taught and in which disciplines.
Putting It All Together
Although still in the early stages of implementing LibPASLibrary Performance Assessment Service, Boyle says she anticipates the system will provide significant benefits to Rutgers, including the ability to easily do peer comparisons in areas such as collection dollars spent per faculty member or number of subject librarians per faculty member. LibPASLibrary Performance Assessment Service also will provide statistical support for fundraising and grant applications. “As we look for new ways to support the work of the library, producing compelling reports to show growth in areas such as online reference and use of special collections like jazz recordings and memorabilia will be important. LibPASLibrary Performance Assessment Service will put data on the desktop to enable library administrators to demonstrate transaction and collection development trends.”
Boyle also can see the potential for linking LibSAT data with LibPASLibrary Performance Assessment Service data to discover new ways to assess performance. “We are looking for efficiency and accuracy so we can begin to collect data beyond what is nationally required. LibPASLibrary Performance Assessment Service is a very flexible yet sophisticated system. I don’t see how it can disappoint. Together, with LibPASLibrary Performance Assessment Service, we have it all in one package.”
About Rutgers University
Rutgers, The State University of New Jersey, is a leading national public research university and the state’s preeminent, comprehensive public institution of higher education. Chartered in 1766 as Queen’s College, Rutgers is the nation’s eighth oldest institution of higher learning and has a centuries-old tradition of rising to the challenges of each new generation. The Rutgers University Libraries are comprised of 28 libraries, centers, and reading rooms located on the Rutgers campuses in New Brunswick/Piscataway, Camden, and Newark, and RU-Online, a digital library. The Libraries provide the resources and services necessary to support the university’s mission of teaching, research, and service and have specialized collections in the book arts, East Asian studies, jazz, labor relations, music, women’s art and leadership, and New Jersey history. The Libraries are distinguished on the Rutgers campuses, and in the region, for their work in copyright education, information literacy, software creation, and institutional repository development. To learn more about the Rutgers University Libraries, please go to: www.libraries.rutgers.edu.
LibPASLibrary Performance Assessment Service for Academic Libraries
LibPASLibrary Performance Assessment Service is designed to provide library management with actionable insight. LibPASLibrary Performance Assessment Service is a web-based, fully hosted subscription service providing real-time, on-demand data access and comprehensive reporting functionality for library customer and performance data.
LibPASLibrary Performance Assessment Service:
- can mirror an organization’s operational and governance model(s), not the other way around. This greatly simplifies implementation across an organization.
- accommodates the integration/importation of data from multiple different sources (both internal and external and either current or historical). LibPASLibrary Performance Assessment Service is neutral in terms of existing electronic solutions (ILS, gate counters, etc.). As long as a system can respond to a query and generate an electronic data output file, LibPASLibrary Performance Assessment Service can accommodate (multiple formats). This functionality overcomes “The Babel Effect” i.e. multiple data silos that are unable to communicate with one another.
- can be deployed horizontally across an institution to meet broader evidence-based management needs. While developed for library needs, LibPASLibrary Performance Assessment Service is equally applicable across all areas involving the capture and management of data.
- is “organic.” Simply put, LibPASLibrary Performance Assessment Service is constantly evolving to meet “real-world” needs. Upgrades, when released, are immediately available to all subscribers at no additional cost.
About Counting Opinions
Facing constant competitive challenges, libraries and library organizations need better tools to understand and manage customer needs and to compete more effectively for scarce resources. In business since 2004, Counting Opinions provides libraries and library organizations with a cost-effective, evidence-based management solutions’ platform for the comprehensive management of their performance and customer satisfaction data. Solutions for custom surveys, open-ended customer feedback, trends, benchmarking, outcomes and peer comparisons are also available.
Note: Interview conducted and case study prepared by JAM Marketing LLC.
Please cite this article as: Rutgers University Library; Jeanne Boyle, Associate University Librarian for Planning and Organizational Research