Academic Library Case Study

Evidence-Based Management

University of Toronto Libraries

Lari Langford, Head, Access & Information, Robarts Library, University of Toronto

With forty-four libraries on three campuses plus various departmental libraries, the University of Toronto Library system – the largest in Canada and the third largest in North America – poses complicated challenges for data collecting and management. According to Lari Langford, Head, Access Services, Robarts Library, the library wanted to have data collected in one place to assess changes over time and improve the way it was measuring the impact of its services.

“It’s important for the library to demonstrate our value to the institution,” Langford said. “We need to measure services and library activities and be able to document and archive our data. To better manage what we were doing we needed a robust assessment management tool.”

The University of Toronto Challenge: Difficulty Sharing Data Among Reporting Libraries

The University of Toronto libraries collect data to respond to various surveys including the CARL and ACRLAssociation of College and Research Libraries surveys and the Ontario Council of University Libraries’ survey to analyze space utilization in academic institutions. Data was also collected for assessing internal risk management.

The library was organizing data input through a legacy web-based system and downloading to Excel each year. The ability to compare year-to-year data was limited to reviewing static PDF documents. There was no way to interact with an archive of the data.

According to Langford, the data being collected was tied to reporting to other organizations. The library wasn’t seeing important non-survey data that various units were collecting such as bibliographic instruction statistics or feedback from focus groups. In addition, the library wasn’t collecting data that would be useful for assessment, including complaint forms and suggestions.

The goal for a new assessment management tool was to allow the library to gather evaluations and demographics related to library service and store that information where it could be accessed for reporting, analysis and planning.

The LibPASLibrary Performance Assessment Service Solution: Improve on Process, Quickly

The library identified LibPASLibrary Performance Assessment Service as a solution for managing, storing and accessing data but with over 1,000 data elements to capture they did not want to take on the project on their own. They turned to Counting Opinions’ consulting services to help them build on and improve on what they were already doing. Although most libraries can manage their LibPASLibrary Performance Assessment Service implementation, in situations where the data collection is very complex and staff don’t have the time or expertise, Counting Opinions’ consulting services are a viable alternative.

One stipulation that made their project more complex was that they wanted to give staff access for data input, but not access to everyone else’s data. And they wanted people to have view-only privileges, not the ability to run reports. “LibPASLibrary Performance Assessment Service gives us the flexibility to manage reporting the way that works for us,” said Langford. “We have 52 unique locations/departments contributing data to LibPASLibrary Performance Assessment Service. We prefer to retain report access at the central library. Once we complete the annual reports we post them to the web so staff at all University of Toronto libraries can see them.”

With the help of Counting Opinions’ consulting services the library was able to implement LibPASLibrary Performance Assessment Service very quickly and immediately began to realize the benefits of using LibPASLibrary Performance Assessment Service for data collection and management.

• Just in time answers to questions. Langford said with LibPASLibrary Performance Assessment Service they can respond to questions they couldn’t answer easily before. For example, to respond to a request for data on interlibrary loan she can compile a report instantly and put it on the web where it is accessible in a matter of minutes. This kind of report would have taken several hours under their old system. “We have a new chief librarian, so it is extremely helpful to be able to pull data quickly to respond to his requests.”
• Ability to analyze data, not just report it. Before LibPASLibrary Performance Assessment Service it was difficult to compare data over time because the data wasn’t stored in one place. It wasn’t possible to use co-relationships to analyze and find significance in the data.
• Easily track data entry points. Langford said with their previous system it was difficult to track gaps in data contribution. Now they can quickly run a report to see which libraries come up with blanks, making follow up much easier. “For managing data gathering for large systems, LibPASLibrary Performance Assessment Service is definitely the answer. In fact, it’s brilliant!”
• Ability to compare data from multiple sources. With LibPASLibrary Performance Assessment Service, everyone measures the same things in the same way. This enables the library to make valid comparisons in terms of staffing, service planning, etc. Circulation data is imported into LibPASLibrary Performance Assessment Service from their integrated library system on a monthly basis to keep data in one central location. This data is pushed out to the libraries through LibPASLibrary Performance Assessment Service.
• Excellent service and support. “Counting Opinions handled everything from the work of rebuilding our system in LibPASLibrary Performance Assessment Service to offering training webinars. They provided all the information I needed to set up accounts and they responded instantly to questions we had,” Langford said.
• Ease of reporting. Langford says she looks forward to smooth sailing with LibPASLibrary Performance Assessment Service now that staff are comfortable submitting data. “We can go into next year without any fuss. Our reports are set up so all we need to do now is change the dates. Building reports for the first time took some work, but going forward producing reports will be fast and easy.”

Looking Ahead

Langford said that the library’s initial goal was to take what they were already doing and make it more efficient. Having accomplished that goal, she said they see a lot of potential for using LibPASLibrary Performance Assessment Service that they haven’t tapped yet. Having gotten through the initial learning curve they are now looking forward to increasing the ways in which they are using the tool.

In addition, the library is looking toward implementing InformsUsForm, Survey and App Builder Service, also Web Form Builder+, a forms tool that will enhance the management of custom feedback (both qualitative and quantitative). The library also plans to implement the LibSatLibrary Customer Satisfaction Management Service module, a continuous customer survey instrument, that permits the capture and management of feedback from library customers. LibSatLibrary Customer Satisfaction Management Service enables the library team to identify opportunities to improve the customer experience, and then assess the impact of subsequent operational changes.

About the University of Toronto

The University of Toronto Library system is the largest academic library in Canada, and is ranked in the top five amongst research libraries in North America. The system consists of over 40 libraries located on three university campuses: St. George, Mississauga, and Scarborough. This array of college libraries, special collections, and specialized libraries and information centres, covers subject areas from architecture to zoology. In addition to more than 16 million holdings, the library system offers access to one of the largest collections of online library materials in North America.

About Counting Opinions

Facing constant competitive challenges, libraries and library organizations need better tools to understand and manage customer needs and to compete more effectively for scarce resources. In business since 2004, Counting Opinions provides libraries and library organizations with a cost-effective, evidence-based management solutions’ platform for the comprehensive management of their performance and customer satisfaction data. Solutions for custom surveys, open-ended customer feedback, trends, benchmarking, outcomes and peer comparisons are also available.

Note: Interview conducted and case study prepared by JAM Marketing LLC.

Please cite this article as: University of Toronto Libraries; Lari Langford, Head, Access & Information, Robarts Library, University of Toronto