Reading Minds: adding value to services at the State Library of New South Wales
The focus of PMs should demonstrate library value and relevance and be the basis for comparison with competitors in other industries.
Performance measurement and benchmarking are two activities that are part of a broader review of services that the State Library of NSW has embarked upon. This review is based on two key components: the Library's Research and Evaluation Program and Quality Initiatives Program. The review is, in part, a reflection of the unprecedented changes in the information environment, as well as increasing client expectations and the growth in remote client use. These changes require a responsiveness that questions definitions of the Library's value and relevance within the current and future information environment.
In our presentation today we would like to discuss some of the ideas and issues that have been raised in our review regarding concepts such as value and relevance in a library context. In particular, our central premise is that while these concepts are two important measures, the critical issue is ascertaining how notions of value and relevance are defined and by whom. At the State Library of NSW we define these concepts from a client perspective. We consider it is imperative that we are aware of the range of perceptions and ideas associated with such concepts, including social and emotive connotations. As these perceptions, often unarticulated, have a major impact on clients' overall perceptions of the Library and the services we offer.
With regard to benchmarking, when considering competitors in other industries, our experience suggests that specific service attributes are a more appropriate basis for comparison than broader concepts such as value and relevance. In this presentation we will outline how we define, measure and benchmark these attributes.
Our presentation will commence with a discussion of the Library's Research and Evaluation Program and what it has revealed about clients' notions of value and relevance. We will then provide a brief overview of some of the services that have been developed in order to meet client expectations, before outlining our approach to benchmarking Library services.
Client research and evaluation
The overall aim of the Library's Research and Evaluation Program is to develop comprehensive information on Library clients - who they are, why and how they use the Library and what they think of current and possible future services. Projects conducted have employed a variety of methods to gather feedback from clients, including in-depth interviews, focus groups and questionnaires.
The projects have highlighted the diverse client base that uses the Library and why it is important to segment clients when considering performance measurement. At the State Library of NSW our clients range from the novice, first time user to the experienced researcher and scholar. And while caution must be exercised in categorising our clients too broadly, they can be segmented into three different, but not mutually exclusive categories: the reading room client, the recreational client and the remote client. This paper will focus on the reading room client.
The reading room client includes both Mitchell Library readers and State Reference Library (SRL) users. While these clients may view an exhibition, attend a Library event, or use our remote services, their primary reason for using the Library is to conduct research in either of the two reading rooms.
In conducting research with our reading room clients, we have discovered the complex, diverse and often contradictory meanings underpinning their notions of the Library's value and relevance. We will now highlight some of these perceptions, before turning to how we have attempted to address and measure them.
Our research suggests that amongst Australian libraries, the Mitchell Library accords 'very' valuable and relevant ratings. This is due to its origins, history and resources. For scholars, particularly in Australian and south - west pacific history, the Mitchell Library is recognised internationally as one of the most extensive sources of original material.
In gathering comprehensive information on Mitchell Library readers for the first time, we have been able to gain a more insightful understanding of who actually uses the Library. In particular, the Mitchell Library provides services to four major reader groups: professional researchers, personal interest readers, students and work/business readers. Each group represents a complex mix of perceptions regarding the value and relevance of the Mitchell Library.
The Mitchell Library's value, according to most professional researchers is defined by a certain culture steeped in the notion of the Library as a place of orderly, scholarly learning and research. Assumptions as to what constitutes a library, reader and valid (serious) research underpin this culture. Inextricably linked to these perceptions is the notion of the Mitchell Library as a constant within a world of change. This projection onto the Library as a steadfast institution representing an unchanging culture and values is, in part, a reflection of significant change in research and academic circles. In particular, many professional researchers feel that the value placed on scholarly research has declined in recent times. The Library, according to these readers is valued and relevant due to its role in providing and maintaining a place and for significant scholarly research. Change in the Mitchell Library is perceived to represent a decline in the value of professional researchers' work, as one reader has remarked:
"There's a concern amongst the historical profession - in these anti-intellectual times - concern that Mitchell Library will lose sight of what it is for".
However, as the research has revealed, not all readers share the views held by professional researchers, personal interest readers are a case in point. Personal interest readers are usually amateur researchers who do not have extensive experience in using the Library (most have been using the Library for less than six years). The majority of these readers find themselves at the Mitchell Library, due to their interest in family history. In general, personal interest readers are highly motivated and very (narrowly) focused on their research. However, unlike professional researchers, these readers define value and relevance in fairly pragmatic terms; the delivery of enhanced services for their specific needs.
Work/business readers share personal interest readers' pragmatic focus. However, there are some additional requirements for this group: the delivery of timely services and increased access to electronic resources via the Library's website.
On the other hand, students represent a shift in research 'values'. The way research is conducted and the approach taken has changed. Many students have departed from traditional subjects such as history and moved to multidisclipinary fields such as cultural studies. Students studying such subjects approach their research from a different, 'postmodernist' framework. The recognition of subjectivity and critical-ficto writing is of more value and relevance to these students than 'traditional' notions of objectivity and academic prose. This shift affects the way students approach their material. In particular, and much to the chagrin of some of their lecturers, not as much emphasis is placed on the use and interpretation of original materials. As a result, students tend to draw from more than one source and therefore do not usually place as much value on the Mitchell Library as their primary place of research.
This brief sketch of reader groups reveals some of the challenges the Library needs to meet in order to deliver valued and relevant services to its different reader groups. However, understanding Mitchell Library readers' perceptions and the ensuing complexities provides an understanding of only half of the picture. As we will now explain, SRL clients hold different ideas regarding the Library's value and relevance.
State Reference Library clients
Apart from students, it is difficult to segment the State Reference Library client. However, while SRL clients avoid neat definitions, they are, in most respects unified in their overall perceptions of the Library's value and relevance. Unlike many Mitchell Library readers, whose sole focus when discussing the State Library is the Mitchell Library, SRL clients tend to take a wider approach and apply their notions of value and relevance to the Library as a whole, rather than just the SRL.
When focusing specifically on the SRL, clients indicate they place the most value on 'efficient' service. Efficient service for SRL clients is defined as being able to use the computer catalogue and retrieve the appropriate items within a certain amount of time. For students, it means being able to photocopy as many references as possible within a set time!
While efficient services are important to SRL clients, they are just as concerned with broader issues associated with the Library's value and relevance. In particular, these clients place significant value on the Library 'experience'. This 'experience' can mean many different things to different clients however, usual definitions are associated with a sense of 'quietness', the pleasure of research and the feeling that they are working within a valued 'community' institution.
Similar to Mitchell Library readers, SRL clients are concerned about change within the Library, albeit from a slightly different perspective. Change for SRL clients is defined by technology and the feeling that new electronic and virtual technology will lead to a loss of control in accessing and organising information. In brief, if we take away the 'physicality' of the Library, in particular, the feeling of touching a book, how does the Library experience remain valued and relevant? As some SRL clients have remarked:
"The future is the net increasing, but you cannot replicate the feeling of touching".
Future concerns aside, SRL clients also need recognition that a place that they 'value' is valued and relevant to the community at large. For while SRL clients are anxious about change within the Library, at the same time, they believe the Library needs to promote and accommodate change in the way that ensures it is seen as a contemporary and relevant institution amongst other cultural institutions and society in general.
As the above research findings highlight, in one reading room alone, the concepts of value and relevance can mean many different things. More importantly, it is not the differences so much that matter, but the tensions that arise in trying to accommodate the various meanings. We will now discuss how we have attempted to incorporate clients' perceptions and expectations within the delivery of reading room services.
Reading room services - the context
A recent article described libraries as offering a "venue that appeals and nurtures every idiosyncrasy" and that staff within these libraries "are apostles of culture"1. This statement is particularly appealing, considering the diversity of the State Library's clients and their requests. But what does it tell us about our value and relevance? How can we measure the way we "nurture every idiosyncrasy" or compare it against the offerings of our competitors?
In the past, the strength of a library such as the State Library was based on its Collections and staff who knew what was in our Collections. Much more is expected of us now. We are still important as a physical entity - a place to visit and see exhibitions or to use the Library's unique collections. However, increasingly the shift is away from collections to services that facilitate access and the delivery of information in whatever form directly to the client, wherever they are. Technology has enabled services to develop in ways that increase client expectations by offering improved choice and convenience. Such changes require a responsiveness that challenges traditional models of reference and research. Our services need to reflect these changes.
This part of our paper focuses on service initiatives, that are a direct result of the client research and the quality initiatives project.
The State Library of New South Wales offers information services from two Reading Rooms: the Mitchell Library which holds the Australiana Research Collection and the State Reference Library which offers both general reference and specialist services such as the Legal Information Access Centre and the Health Information Service. Both Reading Rooms combine to offer information services to remote clients. Recently these two Reading Rooms came under one management structure providing a framework for two different libraries to deliver services in a similar way.
This single organisational unit is a result of staff from both Reading Rooms participating in a Quality Initiatives Project aimed specifically at providing more relevant and effective services to clients. This was achieved by reviewing internal processes and work priorities, analysing gaps in our performance and then developing better practices and performance in our delivery of services. This information coupled with the insights offered by our Research and Evaluation Program has helped us to identify what we have yet to accomplish and how we hope to get there.
As outlined, client research offers us insights into how our clients view what we do. It also helps us understand how our clients seek information, rather than how we think they seek information. This process has prompted us to question our service delivery mechanisms. The traditional ways of providing reference services that have stood us in good stead over the years are now sometimes inefficient and frustrating for both clients and library staff.
Until recently, the Library followed a traditional onsite reference model; one that was anchored at physical service points. Clients queued for both complex and simple directional inquiries at the same point. Such a service symbolised the values of equity and ease of access but it was a one size fits all model.
Another disadvantage was that continuing a consultation process at the desk with the client was very difficult particularly if there were other people queuing behind. Continuity with the same staff member was also difficult because of rostering patterns.
In addition, we recognised that staff were frustrated with this one size model. There were different ways of staffing services and different cultures in the two Reading Rooms - one reading room had a team based service approach, the other was individualised with staff working independently and with very fixed professional and paraprofessional roles. Many of these staff had worked for many years in the same area and had built up strong knowledge of the collections. Regular clients quickly "sussed out" who to approach for ongoing personal assistance, and would wait until their shift. The paradox in this service was that there was a perception of personal service but in reality staff and client were both following a well worn path of compatibility that continued existing beliefs and values.
Our redefinition of services is best described as an iterative, rather than linear process. This approach became apparent to us in planning the "new model" process, which involved many discussions with staff and with our clients, developing their ideas, talking to other libraries, analysing research results and, most importantly, understanding the service from a client perspective though focus group "conversations" with our clients.
These conversations told us that our clients
The results of these deliberations was the development of a new service model - what we called a 3Tier service model, which defines the levels of service that clients can expect from us and the skills and knowledge staff need to provide within these levels of service.
The 3Tier Service Model
The Model provides a framework for service delivery in which we offer a variety of service configurations and differentiated levels of service that meet the broad range of needs and are supported by skilled staff and a more effective use of resources.
Broadly, the 3Tiers comprise:
Reshaping services is not just about changing the way we do things. It must change the outcome for clients. The 3tier model of service only works if it is in a context of value for clients.
For example, what might constitute value for professional researchers? Elements in all 3 tiers can add value. For many, the Library is their workplace so physical spaces, low noise and facilities (space to write, power for their laptops) are very important to this group. This group is usually narrow in their research focus and are often unaware of new services or products, so we provide them with targeted information through a fortnightly newsletter prominently displayed close to where they work. At the other end of the Tier spectrum, offering a consultation with a collection specialist provides a structured arrangement and a guarantee of service to both the researcher and the specialist.
A tiered framework helps us to plan delivery of services to particular groups of clients (e.g. business people, historians, students) in a structured way. Family History clients, like other researchers, use the Library primarily for the strength of resources. Staff understand the complexity of products and resources and can recommend the appropriate search strategy to suit particular needs.
Insight into family historians preferred mode of research, contributed to a slightly different 1st and 2nd tier "merging". This group relishes the common purpose they share with their colleagues. They love to know what their colleagues are doing and what they can learn from them. Many told us they discovered new resources or ways of doing things by talking to other researchers. Taking this into account, we are just about to trial a new service that will include a brief instruction session that will cover new family history resources, or "hands on" technology. Run by staff, the session will continue as a physical "chat room" that will tap into the sometimes club like and serendipitous way family historians work. This allows family historians to draw both on the skills of each other and staff. We are also investigating the possible development of a virtual chat room for family historians.
Other value outcomes relate to reducing complexity and increasing client independence. We recognise the complexities faced by first time users when they visit the State Library. Many struggle to understand and make sense of what they are experiencing but are reluctant to approach the Information Desks to ask for help. They are concerned about sounding like idiots, or wasting the staff's time so try to figure it out for themselves.
A major element of the 1st Tier is the promotion of client independence and self sufficiency. Accordingly, our 1st Tier incorporates a range of products and services from roving staff to electronic and paper based finding aids such as FAQs, self guided tours, introductory tours and "virtual" tours. The value for these clients is in having the guesswork taken out of their introduction to the Library.
We find that a high proportion of questions are simple directional or repetitive questions. (Where are the lockers? Can I join this Library?) The first information source visitors approach are the foyer desks. For many years, senior Library staff worked on one desk and Volunteers on the other. This resulted in patchy service. We realised that it was critical that the responsibility at this desk be well defined and that only directional and general questions be answered. So we introduced a new group of staff known as Information Officers, whose prime purpose was to provide a "front of Library" service that introduced people to the Library and its' facilities. This in turn freed the professional staff to carry out more specialist work and the volunteers to do move in to other support activities
Staff adding value
Personal experience affects how clients perceive the service value of staff. The attitudes of staff, particularly if they value the client's time and deliver what they asked for, are integral to a client's positive experience in the Library.
What attributes ensure effective information professionals? Providing accurate, relevant information is a complex process that requires a blending of information finding skills, interpersonal skills and cognitive processes. An ability to conceptualise information is promoted by some to be the core competency (as opposed to the core skills) that librarians need.2 This means posing the right questions and retrieving only relevant information for the client. It also indicates filtering and evaluating content and saving the client valuable time. Core skills are also important but these change over time. Computer skills are an example of this.
The increasing presence of technology does not eliminate the need for human intervention - rather, in some cases, it has heightened it. Staff are a critical component in helping the client access and navigate an often bewildering information environment. The Health Information Service, for example, has developed a tool to help clients assess the quality of health information on the 'Net. This is in the form of a checklist that covers authority, content, design and relevance. Health staff also utilise other skilled professionals who have specific subject skills (such as the Cancer Council) to assess health sites.
In the 3Tier Model it is not sufficient to focus solely on service outcomes. We need to strongly underpin this service model with adequate support, training, resources and tools. The new model is based on staff having a range of core skills and competencies. Position Descriptions now include elements such as "analytical skills, client focus, communication and interpersonal skills".
William Bridges has made an important point and one that is pivotal to us moving our services forward at the State Library. He draws a distinction between change that is situational (e.g. new team roles) and transitional change which is the psychological process we need to go through to come to terms with new situations.
Collecting the data
Client research and the development of new services highlights the difficulty in establishing consistent measures of the Library's value and relevance. Our experience suggests that quantifying these concepts in a meaningful way can be problematic. Rather, it is a matter of recognising these sometimes 'irrational' attitudes that very often relate to external issues, and the effect they have on clients' service perceptions at the Library.
This being the case, we have developed a Library wide measurement model that takes client satisfaction as its central measure. This model is consistent, yet also flexible enough to incorporate the diversity of Library services.
We define client satisfaction as a composite measure, underpinned by a product, service and if applicable, price factor. We also allow for the 'x' factor, the usually intangible, often 'irrational' element that affects a client's perception of satisfaction. This factor is a broader value judgment or perception that a client brings to the Library, for example a reader's perception that the Mitchell Library is valuable due to its collection. The following model (Figure one) outlines the standard relationship between the variables.
Our research has shown that client satisfaction is a reliable predictor of client behaviour: if a client's satisfaction is high, they are most likely to come again and tell others about the service.
Our focus, when considering a product and/or service is the quality of that product or service. We consider that rating the quality is a far more critical and objective measure than asking a client questions such as 'Did you find the information you require?' While it is important to know such information, questions like this are not a reliable indicator of client satisfaction as they do to not take into consideration the various factors (such as the client's research skills) that can influence the end result. This is why, from a provider perspective it is best to focus measurement on the service we provide.
In some instances we ask clients to consider the overall quality of a service and/or product. However, in order to gain a more meaningful understanding of what a client values and finds relevant regarding a particular service, it is often essential to understand what, for a client, defines the service. The important point is that defining criteria differ from service to service. Without uncovering the attributes for individual services, we run the risk of measuring the wrong attribute, which in turn leads to incorrect conclusions as to a service's performance. A case in point is the measurement of service provided by reading room staff. In the Mitchell Library, the key attribute for readers is that staff provide a consistent service. However, in the SRL, the most important attribute is the friendliness of the staff. To have only measured one of these attributes across both reading rooms would have meant that we were not providing a valid indicator of quality service.
Most importantly, this performance model enables us to identify the priority areas; aspects of a service that have the highest impact on a client's satisfaction rating, yet rank the lowest levels in performance. Highlighting these areas ensures that we dedicate limited resources to the most appropriate areas. Table one highlights the priority areas for the Mitchell Library.
BenchmarkingOur approach to performance definition and measurement means that it will be a while before the Library has comprehensive performance indicators for all of the Library's services. It also indicates that it will be sometime before we are ready to compare ourselves with competitors from other industries.
This is not to suggest that we are not 'benchmarking' our services. In fact we are benchmarking in ways we consider are more appropriate to our current context.
One major component to our internal benchmarking process is measuring a service before changes are introduced and then, once those changes have been implemented and given time to establish, re-measuring the key attributes. The focus of this measurement and benchmarking are specific attributes that define the service, for example, when measuring the performance of the Library's Picture Search Service, the key attributes included the information on the service, the request process and payment. Within these attributes, specific factors were measured, for example, the main criteria for payment was that invoices carried enough information for the client. Knowing which attributes to include and the specific factors defining those variables is uncovered by conducting qualitative research, in this case, in-depth interviews with a selection of clients.
This benchmark activity provides us with not only an objective indicator of the success of new service developments, but very specific information as to the performance of the various aspects of the service. The three tier service model is an example of where we have sought client feedback before the implementation of the service, and where we will ensure client perceptions of performance are measured, once the service is operational.
Another benchmark exercise we practice is comparing key quality measures across services. Comparison enables us to gauge what, from a client perspective, are the Library's quality services. This process also helps us identify 'best practice' services and highlights the attributes that make those services so exceptional.
We also identify best practice competitors by asking clients to tell us who they think provides the highest quality service or product within a particular area. At a later stage, we will be able to use the information in external benchmarking exercises.
At this point in time, we consider all of the above exercises to be of more benefit to us than comparing ourselves to external competitors from another industry. However, once we have established the services we need to provide to clients and have then developed an appropriate number and balanced set of benchmarks, we will consider external benchmarking as a way of ensuring that we reach and maintain the highest level of performance that can be possibly achieved.
1 Nunberg, G. (1998) "Will libraries survive?", The American Prospect online, Issue 41 November- December, http://www.prospect.org
2 Nichols, M., Sikes, J., Isselman, M., Ayers, M., Seelig, R. (1996) "Survival in transition or implementing information science core competencies", Bulletin for the American Society for Information Science, American Society for Information Science, Dec - Jan issue
3 Bridges, W. (1991) Managing Transitions: Making the Most of Change, Reading, Mass., Addison-Wesley Publishing Co.
Berzins, B. (1982) "The Mitchell - a great Australian Library", Australian Library News, Sept/Oct 1982, pg 13
Bridges, W. (1991) Managing Transitions: Making the Most of Change, Reading, Mass., Addison-Wesley Publishing Co.
Burgess, K. (1999) A discussion with Mitchell Library Readers, State Library of NSW (unpublished report)
Burgess, K. (1999) A survey of Mitchell Library Readers, State Library of NSW (unpublished report)
Burgess, K. (1999) "The Mitchell Library: Finite Spaces, Infinite Demands", LASIE, Vol 30, No 4, 1999, pp 68 - 77
Burgess, K (2000) A discussion with State Reference Library and Offsite Information Services Clients, State Library of NSW (unpublished report)
Burns, M. (1998) "Borrowing from libraries", Media International Australia incorporating Culture and Policy, No 89, November, pp 35 - 46
Crosby, L. "Measuring Customer Satisfaction" in Christopher, W. & Scheuing, E (eds) (1993) The Service Quality Handbook, Amacon, New York
Nichols, M., Sikes, J., Isselman, M., Ayers, M., Seelig, R. (1996) "Survival in transition or implementing information science core competencies", Bulletin for the American Society for Information Science, American Society for Information Science, Dec - Jan issue
Nunberg, G. (1998) "Will libraries survive?", The American Prospect online, Issue 41 November- December, http://www.prospect.org
Taylor Nelson Sofres (1999) Mitchell Library readers' research: customer satisfaction service dynamics¨ report, Sydney (unpublished report)
Wholistic Learning (1998) Training notes from "Benchmarking and the role of performance measures", Australasian Evaluation Society Conference, Melbourne
Last modified: 2001-03-18