 home > public2001 > 2001 Public Libraries Conference paper |
|||
2001 Public Libraries ConferenceAccounting for outcomes - demonstrating the impact of public libraries
Professor Bob Usherwood, Department of Information Studies The roles of the researcher and marketer are different but both benefit from the qualitative assessment of the outcomes of public library services. The paper describes how qualitative research data provide images that can be used to demonstrate the value and impact of providing library services. Research at the University of Sheffield suggests that libraries have an import role to play in terms of personal development; social cohesion; community empowerment; local culture and identity; imagination and creativity; health and well-being. These are important messages to be communicated to policy makers and other stakeholders. IntroductionAt a time when so much emphasis is being placed on quality, standards, and accountability, it should not come as a surprise that the academic world is spending time devising new ways of assessing the value and impact of organisations. At the same time, those of you responsible for managing and marketing library services are having to demonstrate their value to the public, policy makers and other stakeholders. However, serious difficulties can arise for researchers and practitioners alike if they try to deal with these issues using inappropriate tools. For example, tools used to demonstrate outputs, may not be the same as those used to demonstrate outcomes. For a number of years we had been considering social audit techniques as a way of evaluating library services. We were influenced by the approach to social auditing developed by the New Economics Foundation, and based on Zadek's idea that the stakeholders in an organisation have the right to be heard. This, says David Boyle (2000) in his splendid book The Tyranny of Numbers, "gives social audits a kind of objectivity which simple market research doesn't have". We discussed these ideas with a number of practitioners and thanks to the support of what was then the British Library Research and Innovation Centre, we were able to undertake social audits of library services in Somerset, a rural county in the South of Britain, and Newcastle, a large city in the North of England. In doing this our aims were to determine how far ideas about libraries are reflected in practice, and to ascertain the degree to which social aspirations are translated into positive achievements. The intention was to provide a framework to enable professionals and policy makers to come to an informed judgement about the value and impact of public library organisations. Value and impact studies are much more than mere measurement. Statistics are just a small part of the reality of the library, and any meaningful demonstration of its value has to go beyond simplistic quantification. Even the quality guru Deming, who is often associated with the use, if not over use of statistics, has admitted that the most important things in life can not be measured. How does one measure love, kindness, and generosity? Many of the measures that are set out in the managerialist literature are often substitutes for qualities that are intangible or indirect. Colleagues in academic libraries have observed that, "relatively little attention has been paid to qualitative measures, or to output measures, but indicators which fail to take such factors into account will be inadequate and misleading." (Joint Funding Councils' 1993) At the same time commentators on the public sector have argued that, "performance can seldom be expressed in a meaningful way by quantitative data only. To a great extent, analysis of performance has to be based on qualitative descriptions and statements." (Arvidsson 1986). Thus we need to use sociological and psychological research skills to demonstrate the value of library services in terms of their impact on individuals, and society. In addition our approach should be concerned with "soft" as well as "hard" data. The management expert Henry Mintzberg (1998) reminds us that "hard information is often limited in scope, lacking in richness and often failing to encompass important noneconomic and nonquantitative factors". Managers, and certainly those involved in marketing also need to know about: "The expression on a customer's face, the mood in the factory, the tone of voice of a government official." As Stewart and Ranson (1988) have observed: "Performance monitoring in the public domain is not merely concerned with effectiveness in achieving stated values, but with unexpected impact, and of values denied." As the title suggests, this paper is concerned with outcomes, and it is presented from the point of view of a researcher. It needs to be made clear that the role of a researcher is different from that of somebody involved in marketing or public relations, and it is important that we do not transgress the boundaries between the different activities. As I said to my friends at the Library Association, who saw promotional opportunities when I undertook my first social audit study, 'If I find that public libraries have no value whatsoever I shall report it.' That having been said, the activities can be complementary and the results of such studies can be used to promote and persuade. In the words of the American journal Marketing Library Services which reviewed our work; "if libraries ...carry out a social audit then ... they could well end up with the kind of images that marketing people could use to communicate ideas to fund holders, members of the public, the media and so on... It's a very powerful technique" (Lyon 1998).In this contribution we shall attempt to share some images with you. As a result of the initial social audit of libraries in Newcastle and Somerset (Linley and Usherwood 1998) we accumulated a large amount of rich evidence, about the role of the public library, and its impact on communities. Evidence that can, and has been, used to demonstrate the value of the public library. The social role of the libraryThe intangible but significant role of the library in supporting the social cohesion of communities was a recurrent theme in both of the studies. It emerged in the urban inner city communities of Newcastle, and in the physically isolated rural communities in Somerset, served either by small static libraries or by mobile services. In both circumstances it appears that the library can help people overcome social isolation and individual loneliness.
In Newcastle, the symbolic value of the library was seen in all the communities visited. The provision and indeed the withdrawal of library services were perceived to convey messages about the local authority's attitude to what one respondent called, "communities under stress." The same was true in rural Somerset where respondents observed that: " would take the heart out of the community. We have few enough facilities as it is...there just wouldn't be anything for the children to do here if the library closed." However the data suggest differences between the impact of the library in different areas. It appears that these are related to factors such as the levels of deprivation and the availability of other resources. It may, for example, be significant that in the most deprived ward, the library was seen as "the heart of the community," at least by library users. OwnershipThere was evidence that this social dimension of library use also encourages a sense of community ownership of the library service. In both Newcastle and Somerset the public library was considered to be a warm and welcoming local institution. A similar situation was found by the Comedia research team in their Cleveland study where: "Library staff had created a warm, almost domestic ambience, a 'home from home,'and this made the library appear to be symbolically owned by - and belonging to - the community, unlike other forms of public provision such as job centres, youth clubs ... and community centres on the same estates." (Greenhalgh,1993) Our social audit study suggests that this is also true of rural areas. In Somerset the library was perceived to be the "community notice board", and a source of community pride. One respondent observed: "just recently I walked past a lady, I suppose she must have been talking to somebody who didn't live in Taunton, and I heard her say, 'have you seen our new library? You must go and see our new library!'" - Elected member There were also some very specific indications of "ownership" in Newcastle. A good example of this was a library which was free from vandalism, although the adjacent school suffered quite badly in this respect. I should be interested to learn if you have any similar examples from your own experience. The educational role of the libraryThere was also strong evidence for the value of the local library's educational role. Our data mirror other research in the United Kingdom and support the view that it is not only the well-resourced central and district libraries that have a key educational role. However until recently the library's educational role has tended to be been in terms of its value to the individual. This can be seen in the special provision being made for independent learners, for example, through the Government funded Open for Learning Scheme, and in the professional debate over the undervaluing of the library's role as a homework centre. What has been less well covered is the potential for the local library, through its educational role, to support community development. Our social audit leads us to believe that there may be an unexplored relationship between the local library's role as a catalyst for personal development and its role in community development itself. In the words of one of our respondents: "the library has the potential to assist the whole process of community development in trying to help people who are disadvantaged, to make the most of what's available to them" - Elected member We would contend that the availability of local resources for personal development is one of the essential ingredients in the creation of a self-confident community. A community that has the skills and capacity to exert influence on the political and economic environment around it. The economic impact of the libraryIn the Somerset case study, the library appeared to have had a discernible economic impact. For example, in supporting local shopping centres that were coming under pressure from the shops in the large town of Taunton. This confirms the relationship between visits to local shops and library use found in some other research. (Sobczyk et al., 1996) In Somerset the library was also seen to be supporting the local tourist infrastructure. The library's economic impact was more difficult to evaluate in Newcastle. This is mainly because, in the wards studied for our research, the emphasis is on the social and community role of the library. However, local politicians and library staff did refer to the value of the business information services. Reading and literacyThe role of the library in promoting reading emerged particularly strongly in Newcastle, where elected members and community co-ordinators were concerned about low levels of literacy and numeracy in the city. The personal experiences recounted by parents and carers attending focus groups suggested that libraries help develop the reading ability of young children, including English language skills in children whose parents'first language was not English. "Every Saturday and Friday I have taken [my daughter] to the library and read some books there for her ... and I borrow a lot of tapes and a video to get her watching and listening...I think without the libraries my daughter's English can't be improved so fast" - Focus Group The data from the interviews and focus groups in Newcastle suggested that the library could be an appropriate base, providing a non-stigmatised environment for specific initiatives on literacy. Developing community confidenceAs we began analysing the data, it became clear that once services are taken up, libraries can be important in developing confidence in individuals and local communities. Evidence for this is provided by a focus group respondent who said: "I know the library helped me a lot when I went to college... I never went to school when I was a kid, but I went to college in 1980 and I just learned a hell of a lot at the library. At one time I wouldn't talk to you like this because I couldn't, but with using the library as much as ... I did it just brought me out you know and that's a good thing as well." The development of individuals' self-esteem can lead to greater community confidence, and there are of course links here with the points already made about "ownership", and the perception of the local library as a positive, warm and welcoming institution, and its contribution to social cohesion. Public libraries are seen as community landmarks that reinforce community identity. In particular we found that local studies services can play a significant part in sustaining community identity. In Newcastle, this record of community life was especially important given the decline in the industries that have traditionally help shape the local regional identity. For many people public libraries are, "the heart of the community". Our respondents, including local politicians saw the local library as having a symbolic value that was regarded by non-users as well as users. One told us: "I think that people see [the library] as a really important local landmark whether they use it or not, and I think if it was withdrawn it would have a terrible effect on people's morale... The impact of it is that it's ... the only thing that people can point to and say that's the... Council which isn't a school ... and in that sense I think it's an important presence of the Council". Elected member Newcastle EquityIt has been argued that, "One test of a democracy is whether it grants equal access to the tools that make knowledge possible" - (New York Times Editorial 16. 11. 98). The need to ensure equity in the distribution of services is one of the factors that distinguish public sector organisations from those in the commercial world. In such circumstances equity is a dimension of the service, which needs to be properly demonstrated. The data from the social audit suggest that library services are usually administered fairly in terms of administrative justice. In terms of equality between groups and communities, libraries were perceived as providing equity for most older people, people with disabilities and those from the ethnic minorities. The equity dimension was felt less by lone parents and unemployed young people. In addition, some groups, especially lone parents tended not to welcome being identified as a priority group. Work in the UK by Roach and Morrison (1998) paints a less positive picture with regard to the minority ethnic communities. This does not necessarily mean that we, or they, are right or wrong. Rather I suggest that it may reflect the position in different authorities and different approaches to management. The demonstration of equity is still in its infancy and there is a tendency to rely to a great extent on numbers. For example, by counting the number of books in a foreign language, or the number of people from a particular community, or background, who use, or do not use a service. Such data are, of course, important. They are, if nothing else, an indicator of an authorities' commitment. However, such figures tend only to deal with inputs and outputs, and here, as elsewhere there is a danger of only measuring what is measurable, and missing what is important about the library service. In assessing equality, people's experience of using the service, and their perceptions of it, can be used to indicate outcomes. Such qualitative outcome indicators are often a more meaningful way of demonstrating a service and its achievements. Management issuesOur data suggest that the extent to which the public library service fulfils its social objectives depends, in some degree, on how the service and the local authority are managed, and also on other factors outside the immediate control of library staff. Factors, which have been identified as helping or hindering the attainment of social objectives, include resources, marketing and awareness of the service, library rules and culture, structure and staff attitudes. Factors less in the control of staff are the location of the library and the fear of crime in the community. We do not, and can not, claim that the public library by itself is responsible for all the life changes that have been reported to us. There are, of course, many other agencies involved and in our research report we make a distinction between intermediate and final outcomes of library use. (Linley and Usherwood 1998 ) However, on the basis of the data we obtained in our original study, it is reasonable to claim that public libraries help individuals and communities "get started" and "keep going" on a wide range of activities. In addition, sometimes with the help of other agencies, public libraries help develop and maintain individual and community development. In short, libraries enrich the lives of many people. The social audit technique makes that enriching process visible. The same can also be true when we borrow research methodologies from other fields. In a recent study, on the value and impact of imaginative literature, we combined the methodology, developed in our previous social audit work, with the "uses and gratification's" techniques used by researchers in the mass media (eg. Brown 1976, McQuail 1994), and the reader response approach which is seen as integral to the understanding of literature. As McKearney (1999) has stated the library profession has "not developed ways of demonstrating the impact of its work with readers [its] impact on individual lives and, on community life and on national priorities". In other words, librarians describe library activities but fail to demonstrate their impact in a way that will persuade politicians and policy makers.
Our research on public library book reading specifically sought to discover the kind of evidence that policy makers require if they, and their political colleagues, are to be convinced of the importance of reading and the value of the public provision of imaginative literature. Local politicians with responsibility for library services were asked: "when planning your local budget for the provision of services, how would you persuade your political colleagues of the importance of the library's provision of imaginative literature to your community?" Most respondents maintained that they did not need persuading. However it was acknowledged that when; Indeed recent evidence (Proctor, Lee and Reilly 1998) suggests that libraries have been closed and it was not so long ago that a British politician asked, "what purpose does reading serve and why should it be provided free of charge to the user?" (Sproat 1993). The politicians interviewed for the reading study placed a heavy reliance on quantitative data. Typically this was provided from a computer system. -{The} Quarterly operations report of the library service gives a snapshot of users and activities at any one particular moment in time. We can get an accurate picture of the proportion of the population who are users, the issue type, number of issues and what type of stock- we can really have our finger on the pulse. We need this to support our investment decisions. [9]. A minority made reference to qualitative techniques that provided feedback from the electorate. These included the PLUS survey, other locally managed surveys, and their personal observation of the service: - We look at the PLUS surveys, comments and requests. Members use the libraries and can see for themselves. We like to see the library being innovative and employing varying forms to encourage the library user. In response to user surveys we have altered the loan period from 4 weeks to 3 weeks, that way they can borrow more books. [2]. However, that having been said, the evidence clearly suggests that qualitative data do not carry as much influence as quantitative data. Most politicians have not gone beyond book issues and still like what they regard as hard facts. By this they mean quantitative measures. At the same time, and confirming McKearney's observation, there is still an uncertainty amongst some professionals about how to demonstrate the value of the library's work with adult readers. One stock manager admitted: "I don't know, we never seem to emphasise reading. Politicians are interested in IT. They like hard facts ... we also stress the library as a safe environment as it is a corporate objective - safety in the community. But we never focus on reading as such. At the launch of Bookstart, I spoke to politicians and parents on the value of getting children hooked on books from an early age and how it helps them in education. But we've never mentioned it at other times." It has been recommended (Nicholson 1987) that we should make arguments about the public library "political". This idea was developed at the workshop organised as an integral part of our reading research. The delegates who attended suggested that it was imperative to capitalise on some current government strategies such as life long learning and social inclusion. Links between the public provision of reading material and the government's concern for literacy were also thought to be important. For example, some recent members of the government have mentioned the importance of reported links between low literacy levels and crime and other anti-social activity. The Moser report (1999) identified that literacy has an impact on a number of levels; those of the individual, the family, the community and society, and the economy. Participants reflected on the individual and social impact of the public library reading experience. Typically their responses included concerns about literacy, the role of fiction and the library's social responsibility. As part of our research into public library book reading we asked library users and other stakeholders how they would explain the value of the public library and book reading to their local councillor. Their answers reflected the broad themes identified by Comedia in their work on the social impact of the arts (Matarasso 1998: iv/v). These are:
Another Sheffield study, on the Impact of library closures and reductions in opening hours (Proctor, Lee and Reilly 1999), involved before and after studies of public library users in communities where closures and reductions were taking place. The data show that when a local library closes up to a third of adults may be deprived of access to the public library. Worst affected are young children and elderly people. Parents and teachers told the researchers that the impact of library closure was devastating. Although some children were reading as much as before, their choice and quality of reading suffered dramatically. People felt that only a local library helped them to feel a part of their own community and play an active part in it. It helped them to be part of a local information "network", reduced loneliness, and encouraged friendships. The Sheffield strike research (Sobczyk et al 1996) provided further information on the value of the public library. In the summer of 1995 Sheffield Libraries closed down for eight weeks. This gave researchers the chance to find out how people responded to the lack of a library service. Over 500 library users were interviewed after the libraries re-opened. Other local libraries and bookshops were surveyed to see whether people had transferred their use there. A survey of telephone callers using the information services was also carried out. The research showed that the social value of the local public library has been underestimated. People enjoy the experience of going to the public library, whether or not they need to borrow books or seek information. It appears to make a significant contribution to their quality of life. This seems especially true in communities with a high incidence of economic and educational deprivation. The local library can be an important resource for personal development, particularly when users have had a poor experience of formal education. The data also demonstrated the extent to which reading is perceived to be an essential and critical factor in the lives of library users. Many people cannot afford to buy enough books to replace those provided through the library service. Perceptions play a significant part in value and impact studies, and research from other areas points to the dangers of making management, or indeed political, decisions on the basis of users' perceptions alone. Simply asking users is often not enough. Users know what they like, they sometimes know what information they need, but they often do not know what is possible for the library to provide. Some explanation of the different perceptions between librarians and users can be found in the literature on the psychology of prediction. For instance Slovic et al (1982) Tversky and Kahneman (1982) suggest that people judge events as likely if instances of it are easy to imagine or recall. What people can recall will depend on their knowledge and experience of the area. Library users by definition will have personal experience of using libraries. They will therefore have some relevant knowledge but as Stewart and Walsh (1989) suggest, from a public service perspective, in a number of areas this knowledge will be incomplete. Demonstrating value and impactThat having been said qualitative assessments of outcomes are often a more meaningful way of demonstrating the value and impact of a service and its achievements. As we have seen, this more qualitative approach, has been used in a number of recent studies of social impact. In Beyond book issues Matarasso (1998) reviewed library based projects entered for the Library Association Community Initiative Award, and assessed the extent to which they have produced social benefits and sought to identify factors which lead to success. The original Sheffield Social Audit study used a technique that had much in common with quality audits, as defined by Percy-Smith (1992), and implemented by a number of local authorities with regard to recreation, transport and information technology services. It has also been used by a variety of voluntary organisations, and companies such as Traidcraft and The Body Shop. To quote Gray (1995) social accounting has experienced "a (long overdue) resurgence as academics ...look for new ways of providing accounts of organisational life." The technique can be used to examine and demonstrate the success or failure of a portfolio of activities and services offered by a particular kind of library. For example inner city, urban or rural branch libraries or those serving designated areas of poverty. It is important that key professional staff and policy makers recognize the value of research as a contributor to, and demonstrator of, library performance. In the UK we have argued the need to establish a centre for the effective dissemination of public library research. We suggested that this should not be confined to work carried out in academic institutions and other research centres but also include the "hidden research" that is undertaken as part of the day to day management of libraries. We have made some suggestions regarding the dissemination of research of interest to the library community. These include establishing a clearing house for the dissemination of reports and surveys prepared by different library organisations; developing a data base of these activities and; establishing a web site which would keep practitioners, academics, policy makers and others up to date with research being carried in library authorities, in universities, and other organisations with an interest in the field. It is a pattern which you might wish to consider for Australia. The 'implementation' of research findings involves more than simply applying a recipe. Research is often about innovation and all innovations require organisational change to some degree if they are to be implemented effectively. For research to be 'implemented', someone needs to know about it, they must influence the organisation and persuade policy makers that change along the indicated lines will be valuable. "Thus whilst knowledge of the results of research may be important, it is rarely by itself sufficient to change practice, and other factors need to be taken into account" (Watt 1996). Senior library managers need to persuade and cajole others in their organisations to try out new things, and to invest in people and other resources to provide the training that changes will require. In other words, implementing research findings is a major organisational task and not something simply to be bolted on. It is about taking a "GRIPP". That is getting research into policy and practice. In this presentation we have concentrated on the positive findings from our research but the approaches we have described also enable us to identify some of the matches, mismatches and differences between social objectives of local authorities, and the intermediate and final outcomes of the library services they provide. The process has also enabled us to identify positive differences between objectives and outcomes. That is to say we have identified beneficial outcomes, which may not have been previously identified as library, or even local authority, objectives. These matters have been discussed with the politicians and professionals responsible for the service and the reasons for the matches, mismatches and differences analysed in terms of the various components of the audit. In particular we believe that some management issues, especially staff attitudes, can help and/ or hinder public libraries in their pursuit of social objectives. The point is that managers, armed with this information, can then take appropriate actions to maximise the matches and minimise the mismatches. In the words of the old song, that seem particularly appropriate given the theme of your conference, they can take action "to accentuate the positive and eliminate the negative" aspects of the service. The methods we have described can be used by staff in public libraries, and indeed other public and voluntary sector information organisations, to help demonstrate the impact of the services they provide, and to enable them to identify the reasons for their success or failure. Currently we are engaged in a research project to investigate the value and impact of cross-domain activities between library, museum, and archives services. An integral part of this project has been to train staff in social audit and other techniques so as to enable them to monitor and demonstrate the outcomes of their services. Much of the work described in this contribution seeks to illuminate the activities of library users. As such it can help professionals and policy makers assess the impact of the services they provide, and enable them to identify the reasons for their success or failure. Such information should, of course, be of use to you who have the responsibility for marketing the library message. In addition, such research can help managers guide and monitor the service, and improve the way outcomes of the service are reported to policy makers. It should enable stakeholders to make better judgements about the service, and affect organisational behaviour. In short it has a practical worth. Not the least it can provide professionals, and marketing experts, with data to demonstrate the value of the public library. References
Arvidsson, R (1986) Performance evaluation In Kaufman, F X, et al eds. Guidance, control and evaluation in the public domain. Berlin. de Gruyter
|
|
