Cheap, Useful, Fairly Valid –
Next Generation Statistics
Derek Whitehead
CAUL
January 2003
Summary of the Session
1
Why statistics are useful / not useful
2
Why cheap, useful, fairly valid?
3
What library statistics should do
4
Some random examples
5
Issues for CAUL/CONZUL
2
Why Statistics are Useful

Determine how well we are going in relation to like institutions
and to ourselves, over time

Understand and improve service delivery

Provide a basis for resource allocation and budgeting

What isn’t counted isn’t valued

Demonstrate extent and nature of complex changes over time

Provide a more objective backing for judgement and opinion

Enhance understanding of customers and their demands

Opinion/user data provides an objective insight into the customer

Statistics are an important way to communicate upwards
3
Why Statistics are a Waste of Time

They can be extremely time-consuming – libraries put more
resources into them than anyone except Finance – look at ITS

They can be spectacularly inaccurate and meaningless –
reference statistics, for example

They can be extremely misleading

Comparisons almost always have flaws because of the difficulty
of comparing like and like in complex service environments

Much statistical information collected is NEVER used

Opinion/user data is notoriously unreliable

Consistency is harder to achieve than you think

No-one believes them.
4
Cheap, Useful, Fairly Valid
Why these three characteristics?

Library statistics are a part of the cost of managing,
nothing more – keeping cost/effort down is essential.

Useful should go without saying, but it is worth saying –
if you don’t use the data, why keep it?

Fairly valid is a corrective both ways – you need SOME
level of validity, but you can go too far.

. . . and of course, excessive validity can undermine the
usefulness of statistics
5
What Library Statistics Should Do
Statistics which universities keep and make available
could do the following a lot better than they do:

Give an idea of actual outputs

Indicate changes in type/balance of outputs

Convey what we do

Relate what we do to what others do – benchmarking

Understand the customer better

Help us to plan

Provide very large numbers
6
Some Other Statistics

Photocopying and printing – copies

Active borrowers

In-house use

Catalogue searches

Online reserve downloads

Student online sessions

Loans – self-service to assisted ratio

Library website hits

CAUL E-Metrics survey: Electronic resources; Use of electronic
resources; Expenditure; Ratios
7
Some large numbers
Data from various years, 2000/2002

Library website hits
2,511,659

Catalogue searches
695,200

Loans (including renewals)
565,786

Photocopies and prints

Documents downloaded

Traffic (turnstile)

In house use of collections
2,229,000
NA
1,196,223
300,000
8
Example: Who uses the library?
Number who have borrowed one or more item from the
library over the preceding 12 months (“active
borrowers”):

Higher Education students

TAFE students
40.39%

Academic staff
63.49%

Other
63.76%

TOTAL
58.03%
80.12%
9
Example - HEEFTSU

Cost drivers allocate funds from TAFE/Higher Education
and based on
 Loans/info literacy classes formula
 survey

EFTSU counts Higher Education students

FTE counts TAFE students (720 SCHs = 1 FTE)

HEEFTSU combines TAFE and Higher Ed students
 A precedent in copyright – 100:22
 Swinburne ratio – 100:38
10
Example – “hits”

Here is some information on hits. Or we could use
downloads, sessions or searches.
Total hits
2,511,659
Less php hits
-679,450
Less menu hits, images, temp
-113,020
Net of misc. hits
1,700,730
Less other directional
-901,661
Total
799,069
11
Hits, what hits?
Most downloaded file types

Htm
1,101,831

Php
683,312

Gif
255,949

Html
136,945

Pdf
112,103

Jpg
20,875

Swf
8,507
TOTAL
2,334,868
12
Example: mobile phones
Question: Do you think that students should be allowed to
use MOBILE PHONES in the library?

Never
1,691
31%

Yes, but only in non-silent areas
1,142
21%

Yes, but must ring and talk quietly
1,197
22%

Yes, any time
1,458
26%

TOTAL
5,488
100%
13
Example – “reference” questions
3453 in-person questions over a week, 100,000 per year

Directional – university
105 (3.0%)

Directional – library & hours

Referrals (all)

Service enquiries & bookings

Photocopying and printing
395 (11.4%)

Other IT-related enquiries
381 (11.0%)

Catalogue enquiries

Other information enquiries
347 (10.1%)
91 (2.6%)
1275 (36.9%)
255 (7.4%)
520 (15.1%)
14
Issues for CAUL
1.
The deemed list – unique in the world – does it work,
is it useful?
2.
The current statistical set – is there too much? are
some things just included because they exist?
3.
Are there missing statistics? – data we need, don’t
have
4.
If so, which statistics should we add?
5.
Is the presentation format adequate, or could it do
more for us? – see www.arl.org./stats
15
16