SYNTHESYS Performance Indicators for Natural History Collections Management.
Michel Guiraud, MNHN Paris and Jonathan Gregson, NHM London
SYNTHESYS is a European Union-funded Integrated Activities grant which aims to create an accessible, integrated European
resource for researchers in the Natural Sciences. Networking is one of its three main activities, and one of the Networking
Activities (NA2.1) aims to implement best practice benchmarks in collections care, to raise standards and improve accessibility to
collections for all physical Users.
One of the objectives of NA2.1 was to deliver Performance Indicators (PIs) in collections management. The traditional measures
supplied to funding bodies etc. such as numbers of loans, are limited in their value for determining genuine change or attainment
of a benchmark in collections management. The aim was to produce a set of transferable collections management PIs and
definitions. They have been designed to allow institutions to create their own subset of indicators for internal management,
which will prove useful in developing new, more efficient ways of working. A workshop was held in October 2011 to develop the
PIs, and to give an opportunity for other museums and science sector organisations to have a two-way exchange of ideas, and to
seek new methods to improve workflows and processes to maximise the efficient use of staff time and other resources.
What are Performance Indicators?
Since the 1980s, museums and other public-sector institutions, where performance cannot be measured simply by profit, have
developed performance indicators (PIs) for use by museum managers, funding bodies and regulators to demonstrate
accountability, efficiency and effectiveness.
Collections managers often use PIs in this context, rather than to inform policy and planning in collections management, for
which these traditional measures are often of limited value.
How can PIs help?
Areas where performance measurement contributes to museum management activities include:
1. Formation and implementation of policy
2. Planning and budgeting of service provision
3. Service standards
4. Efficient use of resources
5. Control and influence over decision making
Collections managers can also use PIs to identify how performance has changed over time, compare departments within an
institution or between similar museums.
The process of determining which areas of collections management should be monitored, and setting standards, has the
potential to encourage discussion of what collections management should be, and what qualities all museums in the same sector
(e.g. Natural History) should have, leading to museums within that sector to unite and be strengthened under a common goal.
The process of developing targets and the ability to measure performance should clarify the mission and sense of purpose of
collections managers, provide evidence of their commitment to excellence and improve their management systems.
What makes a good PI?
Consistency and comparability. Definitions consistent over time and between institutions to enable comparison of like with like
Clarity. Simple, well-defined, easily understood
Controllability. Only aspects of performance over which there is control should be measured
Limited. Concentrate on a limited number of PIs that give the most valuable collections management information
Feasibility. Can the PIs be measured easily?
What are the risks?
Measurement
•
PIs from different institutions or departments must be comparable: definitions of units of measure and rules must be
exactly the same.
Behavioural
•
PIs must be kept in perspective to prevent the focus of collections management switching from actual mission
accomplishment to the PI system itself!
•
PIs must help enough to be worth the effort.
•
PIs should not focus too much on the managerial control or appraisal aspects, or performance measurement could be
regarded by staff with apprehension or hostility.
Conceptual
Many of the qualities of good museums cannot be measured numerically, for example PIs cannot easily measure importance or
quality and should not be read as more than a partial portrait. At best, they can only identify characteristics of efficiency that
lend themselves to numerical analysis, and give some indication of whether a collection is well managed.
SYNTHESYS Collections Management Performance Indicators for Natural History Collections
The SYNTHESYS Performance Indicators (PIs) have been assigned to four categories, relating to:
long term trends in collections
distinct collections management activities
and:
the level to which activities are performed
the efficiency with which activities are performed
Long term trends
Activity level
Efficiency level
Distinct activities
1
3
2
4
A total of 68 indicators are suggested.
Words in italics are defined in the glossary at the end of the table.
To compare departments within an institution or to compare different institutions, most of these indicators
can be compiled for any size of collection unit, and could be standardised, for example by dividing by the
(estimated) number of specimens in that collection unit (PI 1.1).
Performance Indicator
1. Collections
1.1 Number of specimens in collection.
1.2 Number of type specimens in collection.
1.3 Number of species represented in collections.
2. Access and Use of Collections
2.1 Number of requests for specimens on loan to other
institutions for research in one year.
2.2 Average time to process approved loan requests
(working days) over one year.
2.3 Number of requests for digital images in one year.
2.4 Number of requests for specimens on loan for exhibition
in one year.
2.5 Number of requests to visit the collection in one year.
2.6 Number of destructive sampling requests for DNA
extraction/other purposes in one year.
2.7 Number of other requests.
2.8 Number of research loans sent in one year.
2.9 Number of specimens (or curation units) sent on loan to
other institutions for research in one year.
2.10 Number of outgoing research loans returned (or
partially returned) in one year.
2.11 Number of specimens (or curation units), loaned for
research, returned in one year.
2.12 Average length of research loan transaction.
2.13. Percentage of research loan transactions completed
within 3 years.
2.14 Number of EU institutions to which research loans are
in place at the beginning of each year.
2.15 Number of non-EU institutions to which research loans
are in place at the beginning of each year
2.16 Number of institutions from home country to which
research loans are in place at the beginning of each year.
What is it an indicator of?
Category Comments
Collection size.
Type richness.
Scope of collection.
1
1
1
Interest in the collections by scientific
community for visual examination at their
home institution.
Outgoing loan turnaround time.
2
Interest in digital images of the collections.
Interest in the collections by museum
exhibition community.
Interest in the collections by scientific
community for visual examination at the host
institution.
Interest in the collections by scientific
community for testing beyond visual
examination.
Interest in the collections for other purposes.
2
2
Comparison with requests shows loan refusal
rate (for any reason).
Collections management activity (outgoing
loans).
Loan return rate, to compare with loan
request rate.
Collections management activity (return of
outgoing loans).
Efficiency of loan management.
2
Efficiency of loan management.
3
Spread of use of collections within Europe.
1
Spread of use of collections outside Europe.
1
Spread of use of collection in home country.
1
4
Estimated, based on SYNTHESYS collections survey. Updated each year.
Estimated, based on SYNTHESYS collections survey. Updated each year.
For each taxonomic group. Updated each year.
Mean for all approved loan requests of the difference between the date loan
request was received and date of loan despatch (in working days), over one year.
Includes both internal and external requests, recorded separately.
2
2
From both internal and external researchers. Sampling for DNA extraction and for
other purposes counted separately.
2
Includes requests to visit for non-scientific purposes, collections enquiries etc. but
not identification requests.
Number of approved loan transactions.
2
2
2
3
Mean for all returned loans of the difference between the date loan was requested
and the date of complete return (loan transaction completed), for all loans
returned over the past year.
Performance Indicator
What is it an indicator of?
2.17 Number of specimens (or curation units) sent on loan
to other institutions for exhibition each year.
2.18 Number of specimens on loan for exhibition at the
beginning of each year.
2.19 Number of scientific visitor days in one year funded by
SYNTHESYS or other „mobility‟ programmes or internal
schemes.
2.20 Number of scientific visitor days in one year not funded
by SYNTHESYS or other „mobility‟ programmes or internal
schemes.
2.21 Number of scientific visitor days (visitors from home
country) in one year.
Collections management activity.
2
Usually involves more time and paperwork than research loans.
Number of specimens visually accessible to
the public.
Physical access to the collections through
funded programmes. Usefully compared to
access via loans.
Physical access to the collections not
through funded programmes. Usefully
compared to access via loans.
Physical access to collections to scientific
visitors from home country. Usefully
compared to access via loans.
Usefulness of collections to scientific visitors.
1
Include internal and external exhibition loans.
Scientific diversity of users of collections.
1
Physical access to the collections for nonscientists.
Use of collections by scientific community for
testing beyond visual examination.
Number (and therefore proportion) of
collection linked to accessible DNA.
2
Includes historians, artists, schools, VIPs, fund raising etc.
2
Both internally and from other institutions.
1
Number of DNA bank numbers returned in one year.
Use of institution‟s labs for extracting
DNA/RNA from specimens in institution‟s
collections.
Use of collections in published scientific
research.
2
2.29 Total number of specimens accessible via the internet
through species lists or collections-level descriptions.
Digital accessibility: low level.
1
2.30 Total number of records accessible via the internet in
searchable databases where users can browse individual
specimen records.
2.31 Total number of records accessible via the internet in
downloadable data files for a selection of records defined by
the user.
Digital accessibility: medium level.
1
Digital accessibility: high level.
1
2.22 Average length of visit for scientific visitors over one
year.
2.23 Proportion of taxonomic or systematics
researchers/non-taxonomic researchers (scientific visitors)
in one year.
2.24 Number of non-scientific visitor days in one year.
2.25 Number of specimens sampled for DNA
extraction/other purposes in one year.
2.26 Cumulative total number of DNA/RNA extractions from
specimens from collection, which are known to be lodged in
a DNA bank.
2.27 Number of DNA samples (aliquots) derived from
specimens in collections supplied for institutional
use/external use.
2.28 Specimen citation index.
Category Comments
2
2
2
1
1
Median (and quartiles) duration of all scientific visits (recorded to nearest half day)
over one year.
Number of articles from an agreed list of peer-reviews journals (to be agreed on)
citing specimens from an institution‟s collections in one year. Investigate funding
to undertake this centrally for all SYNTHESYS institutions.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens. Updated
each year.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens. Updated
each year.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens. Updated
each year.
Performance Indicator
What is it an indicator of?
2.32 Total number of digital images linked to specimen
records accessible via the internet to a minimum agreed
quality standard.
Digital accessibility: images.
Category Comments
1
2.33 Number of website hits/searches/downloads in one
year.
3. Digitisation
3.1 Cumulative total number of records in electronic
collections database.
3.2 Cumulative total number of specimen (or curation unit)
records with only core fields completed.
Use of digital collections.
1
Collections management activity: databasing.
1
Depth of digital record data: low level.
1
3.3 Cumulative total number of specimen records with full
record data (not georeferenced).
Depth of digital record data: high level.
1
3.4 Cumulative total number of georeferenced specimen
records.
Depth of digital record data: spatial data
added.
1
3.5 Cumulative total number of digitally imaged specimens
to a minimum accepted quality standard.
Progress in digital imaging.
1
3.6 Number of individual collections databases used by
institution.
4. Collections management and conservation
4.1 Collections storage area surveyed to assess their status
and condition and identify potential risks in the past year
(and cumulative total over last three years).
4.2 Percentage of specimens surveyed in the past one/three
years identified as damaged or deteriorating and requiring
conservation treatment.
4.3 Collections storage area with environmental conditions
monitored.
4.4 Collections storage area with environmental conditions
controlled within agreed environmental parameters.
Database integration.
1
Includes molecular collections database entries, but counted separately. Updated
each year.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens. Updated
each year.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens. Updated
each year.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens. Updated
each year.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens.
Minimum standard to be decided (e.g. 300 dpi JPEG and associated record with
at least core fields complete. Updated each year.
Should ideally be minimised. Updated each year.
Monitoring of collection condition.
1
Expressed as percentage of total collections storage area. Updated each year.
Collection condition.
1
Based on estimate of number of total number of specimens in collections storage
area surveyed.
Environmental monitoring.
1
Expressed as percentage of total collections storage area. Updated each year.
Environmental standards.
1
4.5 Collections storage area with storage furniture meeting
established standards.
4.6 Total number of specimens given conservation
treatment in one year.
Collections storage furniture standards.
1
Conservation activity.
2
Expressed as percentage of total collections storage area. Parameters will vary
depending on the type of material stored. Use SYNTHESYS standards? Updated
each year.
Expressed as percentage of total collections storage area. Use SYNTHESYS
standards? Updated each year.
Difference between no. of specimens identified as requiring treatment, and
number receiving treatment each year will identify potential conservation backlog
developing.
Count type specimens and non-type specimens separately. Can also be
expressed as % of (estimated) total number of type/non-type specimens.
Minimum standard to be decided (e.g. 300 dpi JPEG and associated record with
at least core fields complete. Updated each year.
Updated each year.
Performance Indicator
What is it an indicator of?
4.7 Collections storage area covered by security procedures
and guidelines operating to agreed standards.
4.8 Collections storage area covered by disaster/salvage
plan operating to agreed standards.
4.9 Number of new accession transactions in one year.
4.10 Total number of specimens accessioned in one year.
Security.
1
Disaster planning.
1
Rate of accession activity.
Rate of growth of collections.
2
2
4.11 Total number of “gifts-in” specimens in one year.
4.12 Total number of “gifts-out” specimens in one year.
4.13 Number of specimens prepared in one year.
Level of incoming specimen donations.
Level of outgoing specimen donations.
Collections management activity:
preparation.
Backlog size and trend
(increasing/decreasing).
Collections arrangement.
2
2
2
Collections management activity: research.
2
Includes papers on collections management, or collections-based research.
Identification/naming activity.
2
Indication of value added through loans,
visitors, digitising projects etc.
Level of identification.
2
Expressed as percentage of total specimen records, as a proxy for total number
of new/revised identifications across the collections which might be too difficult to
measure
Number of type specimens in the collection, which have not previously been
recognised or marked as such, identified each year.
Updated each year. Level of identification across whole collection too difficult to
measure; this will provide an estimate.
1
6.2 Total spending on conservation.
Collections management: financial
resourcing.
Conservation: financial resourcing.
6.3 Total number of FTE collections management staff.
Collections management: staff resourcing.
1
6.4 Total number of FTE conservation staff.
6.5 Total number of FTE collections management and
conservation volunteers.
6.6 Percentage of collections management/conservation
staff receiving training in one year.
6.7 Percentage of working time collections
management/conservation staff attending training.
Conservation: staff resourcing.
Volunteer activity
1
1
Staff training level.
1
Staff training time.
1
4.14 Total number of unprepared specimens (backlog).
4.15 Average time for collections management staff to
locate/retrieve any specimen required.
5. Research/scientific activities
5.1 Number of papers published by collections management
staff in one year (average per FTE staff member).
5.2 Number of determination fields in specimen database
updated in one year.
5.3 Number of type specimens newly recognised in one
year.
5.4 Percentage of records in specimen database identified
to species level or above.
6. Resourcing
6.1 Total spending on collections management.
Category Comments
1
1
1
1
Expressed as percentage of total collections storage area. Use SYNTHESYS
standards? Updated each year.
Expressed as percentage of total collections storage area. Use SYNTHESYS
standards? Updated each year.
Also expressed as percentage of total number of specimens. Includes silica
gel/chilled/frozen collections (counted separately?).
Also expressed as percentage of total number of specimens.
Also expressed as percentage of total number of specimens.
Also expressed as percentage of total number of specimens accessioned.
Equals historic backlog PLUS number of new specimen accessions in one year
MINUS number of specimens prepared in one year.
Write a procedure to get this number. Tested on an annual basis. Target should
be within 15 minutes?
As a proportion of overall budget. Excluding salaries of permanent/fixed-term
staff. Including contractors.
As a proportion of overall budget. Excluding salaries of permanent/fixed-term
staff. Including contractors.
As a proportion of total FTE staff for institution. Includes permanent/fixed-term
staff.
Includes permanent/fixed-term staff.
As a proportion of total FTE collections management and conservation staff for
institution.
Glossary
Accessioning. The final stage in the acquisition process, when specimens are legally deposited into the institutions
collections.
Collection = collection unit.
Collection Unit. Any defined set of specimens. E.g. lichen collection, Botany department or whole institution.
2
Collections storage area. Area containing collections, in m .
Conservation treatment. Any interventive treatment of a specimen to minimise the risk of deterioration to the
specimen or their associated information.
Core fields. Unique identifier (barcode), Collector, Collection number, Date, Determination/Identification, Country.
Fields filled with “unknown” or s.n. if data unavailable on specimen. The core fields parameters may vary from one
collection to another, e.g. collector may be an important core field in Botany but not in Entomology.
Curation Unit. The unit used for administration purposes when recording loans etc. Can be an individual specimen,
or a group of specimens e.g. a herbarium sheet or diatom slide.
Full record data. Core fields plus all other data associated with and found on the specimen (complete locality,
habitat, description, determination history etc.). The number of records with full data will be difficult to measure, a
better approach may be number of records containing more than core field data.
Georeferenced. Record includes latitude and longitude (as accurate as specimen locality data allows).
Gifts in. New acquisitions acquired through donations from other institutions.
Gifts out. Duplicate specimens and unwanted acquisitions donated to other institutions.
Loan transaction. Complete outgoing loan process from receipt of loan request to complete return of all specimens.
Each loan transaction step (receipt of request, partial dispatch, full dispatch, partial return, full return) should be
logged.
Non-taxonomic researchers. Users of the collections for purposes other than systematics or taxonomy (including
molecular systematics), e.g. ecology, population studies, climate change research etc.
Record. A single entry in the institution’s specimens database (not necessarily a specimen record, it could be a
curation unit e.g. entomology tray or a collection unit (collections-level description).
Requests. The number of requests for visits, loans, destructive sampling etc. are counted even if the request is
subsequently not approved or no material is available. The list of requests to be considered is:
Visits (by scientists for research purposes)
Loan for research: covers all requests which are simple with respect to administrative treatment (little
paperwork)
Loan for exhibition: covers all requests that are complex with respect to administrative treatment (lot of
paperwork: facility report, condition survey, insurance etc.).
Destructive sampling
Other (anything not in the above, includes vests for PR purposes, school visits, collections enquiries etc.
but not identification requests).
Specimen. A unique object or portion of an object, accessioned into the institution’s collection with associated data.
Visitor days. Total number of scientific visitors each day, to the nearest half day.