Digital Services Team Recommendations
October 2010
Submitted by
Karen Estlund (University of Oregon), chair Marita Kunkel (Pacific University) Alex Merrill (Washington State University) Ann Lally (University of Washington) Anne Frantilla (Seattle Municipal Archives) Joanna Burgess (Reed College) Michael Klein (Oregon State University) Michael Paulus (Whitman College) Mike Spalti (Willamette College) Kyle Banerjee, Digital Services Program Manager, staff liaison Jodi Allison‐Bunnell, Northwest Digital Archives Program Manager, ex officio ...with significant support from Isaac Gilman (Pacific University) TableofContents
Charge ............................................................................................................................................. 4 Summary of Recommendations ..................................................................................................... 5 Code4Lib NW .............................................................................................................................. 5 Storage Services .......................................................................................................................... 5 Preservation Training and Policy ................................................................................................ 5 Hosting Systems / IR ................................................................................................................... 5 Scanning / Reformatting ............................................................................................................. 5 Cross Search Utility (formerly Discovery Prototype) .................................................................. 6 Future Directions ........................................................................................................................ 6 1. Code4Lib NW .............................................................................................................................. 7 Recommendation and Proposal ................................................................................................. 7 Summary of Activities ................................................................................................................. 7 2. Preservation Storage ................................................................................................................... 8 Recommendations ...................................................................................................................... 8 Summary of Activities ................................................................................................................. 8 Cloud Storage .......................................................................................................................... 8 DuraCloud Pilot ....................................................................................................................... 8 Curate Camp ........................................................................................................................... 9 Offline Storage ........................................................................................................................ 9 3. Preservation Training and Policy .............................................................................................. 10 Recommendations .................................................................................................................... 10 Policy Summary of Activities ..................................................................................................... 10 Training Summary of Activities ................................................................................................. 10 4. Hosting Systems / IR ................................................................................................................. 12 Recommendation ...................................................................................................................... 12 Summary of Activities ............................................................................................................... 12 5. Scanning / Reformatting Recommendations ............................................................................ 13 Recommendation ...................................................................................................................... 13 Summary of Activities ............................................................................................................... 13 6. Cross‐Search Utility (formerly Discovery Prototype) ................................................................ 14 Recommendation ...................................................................................................................... 14 Summary of Activities ............................................................................................................... 14 7. Future Directions ...................................................................................................................... 15 Recommendations .................................................................................................................... 15 Digital Services Day ............................................................................................................... 15 Organizational Structure within Alliance .............................................................................. 15 Process to Review New Ideas / Strategies ............................................................................ 15 Appendices .................................................................................................................................... 16 Appendix A. Cloud Storage Proposal ............................................................................................ 16 Appendix B. Cloud Based Sandbox Service (Pilot) ........................................................................ 17 2
Appendix C. Distributed Offline Storage Pilot .............................................................................. 19 Appendix D. Digital Preservation Services Planning Proposal ..................................................... 21 Executive Summary ................................................................................................................... 21 Recommendations .................................................................................................................... 21 Background ............................................................................................................................... 21 Appendix E. Trustworthy Archives Audit and Certification Checklist: Administrative Section .... 30 Appendix F. IR Hosting (Digital Asset Management) Proposal ..................................................... 52 Appendix G. Findings of Digital Asset Management Systems Evaluation .................................... 54 Power Solutions ........................................................................................................................ 54 Traditional Institutional Repositories ....................................................................................... 54 Journal Publishing ..................................................................................................................... 55 Multimedia ................................................................................................................................ 55 Consortia Projects ..................................................................................................................... 56 Appendix H. Cross Search Utility Proposal ................................................................................... 58 Appendix I. Visual Description of Cross Search Utility .................................................................. 62 Appendix J. Textual Description of Cross Search Utility ............................................................... 66 3
Charge
January2010
The Digital Services Team (DST) is charged with investigating, developing, and providing recommendations concerning collaborative digital services. Initiatives that have a broad potential budgetary or service impact are developed as recommendations for EC and Council consideration. Selected smaller projects do not require Council action but are presented as service options (opt in/out) subject to EC approval prior to launch. Each service proposal should address all topics relevant to beginning and sustaining a new service, such as start‐up and recurring costs, distribution of fees, likely participants, technology, hardware depreciation, documentation needs, and how oversight and coordination are achieved on a continuing basis. Approved without further review ● Code4Lib NW DST will advise Alliance staff regarding the hosting of Code4Lib NW on a cost recovery basis (i.e., Alliance staff time contributed; no Alliance funding). Service Options for EC approval The following initiatives as described in the Institutional Repository Task Force report of October 9, 2009 and Digital Program Working Group report of September 8, 2009 will be developed as proposals and forwarded to EC for approval prior to launch. ● Cloud Storage ● Offline Storage ● Preservation training Recommendations for EC and Council The following initiatives as described in the Digital Program Working Group report of September 8, 2009 will be developed as recommendations forwarded to EC and Council. ● Institutional Repository DST is encouraged to consider a range of technologies (e.g., DSpace, ContentDM, Fedora) and hosts (e.g., member, Alliance, other consortium, vendor). Examples include but are not limited to WSU hosted DSpace, UW hosted Content DM, Colorado Alliance hosted Fedora, and vendor hosted IR. ● Scanning & reformatting DST is encouraged to consider a range of hosts (e.g., member, Alliance, other consortium, vendor). ● Discovery prototype (now called Cross Search Prototype) DST will consult with project staff from related efforts (e.g., University of Illinois at Urbana‐
Champaign) and develop recommendations concerning appropriate Alliance activities. 4
SummaryofRecommendations
Code4LibNW
The Digital Services Team recommends the Alliance provide continued endorsement and support for an
annual Code4Lib NW conference. Registration will continue to cover costs. The Alliance Staff should
continue to support registration, scheduling, and logistical support at an estimated 40 hours. A planning
committee will coordinate communication, proposals, and speakers.
StorageServices
•
•
•
•
Expand availability of cloud servers as systems testing and production environments
Expand Offline Storage project to include more institutions
Continue investigation of preservation storage solutions including: California Digital
Library Microservices, iRODS, DuraCloud, etc. Complete participation in DuraCloud
pilot for further review of cloud storage systems and preservation systems.
Discontinue offering cloud storage as a product. If demand later emerges for this service,
it can be re-evaluated.
PreservationTrainingandPolicy
●
●
●
Create a Digital Preservation Service within the Digital Services Program to create a system, workflow, and policy to ensure long‐term access to digital objects and work toward a trusted digital repository Complete the following specific activities in 2011: ○ Assemble a Digital Preservation Service Planning Team, composed of individuals from the Alliance and NWDA with key skills (See full planning document for details) ‐ January 2011 ○ Assemble a small group of institutions to form the pilot group for the program ‐ June 2011 ○ Conduct a risk assessment of pilot group’s materials and an analysis of the skills and resources needed ‐ Summer/Fall 2011 ○ Articulate the skills and resources by the end of the calendar year ‐ October 2011 Coordinate preservation training concentrating on new and emerging areas of digital preservation led by the Digital Services Program Manager HostingSystems/IR
To create the most flexible solution, the Alliance should host a Fedora Commons instance with one or more interfaces such as: Islandora or Hydra/Blacklight. (Fedora does not have a native access interface.) Streaming support should also be investigated for viability during the pilot phase. The new IT Manager would spend approx. 3 months to set‐up and test system before production with two pilot institutions. Ongoing systems support to be provided with .25 FTE of the IT Manager and .25 FTE of the Digital Services Program Manager for training and support. Scanning/Reformatting
The Digital Services Team recommends the Digital Program Manager continue to update public list of vendors and act as a liaison for member institutions with concrete needs to help identify potential partners with similar needs for cost savings. 5
CrossSearchUtility(formerlyDiscoveryPrototype)
The Digital Services Team recommends that the cross‐search utility (formerly known as the discovery prototype) initiative move forward to pursue an implementation grant from the Institute for Museum and Library Services under the leadership of the Northwest Digital Archives (NWDA) program. This initiative is of great interest to the NWDA membership and has vital importance for the NWDA program's continued sustainability. The NWDA steering team endorses this recommendation and is willing to lead this project and incorporate any viable outputs into its current services and systems. The IMLS National Leadership grant, due February 1, 2011, if awarded, would allow work to begin on the cross‐search utility as early as October 2011 and can be for between $50,000 and $1,000,000 for up to three years. FutureDirections
●
●
●
Creation of a Digital Services Day to promote community, sharing of ideas, and communicating Alliance activities to the membership, based on the successful Summit Day model. Creation of a small Digital Services Steering Team with strong Alliance staff support and small working groups devoted to particular tasks Creation of a procedure to propose new ideas and areas of investigation from individuals at member institutions 6
1.Code4LibNW
RecommendationandProposal
The Digital Services Team recommends the Alliance provide continued endorsement and support for an annual Code4Lib NW conference. Registration will continue to cover costs. The Alliance Staff should continue to support registration, scheduling, and logistical support at an estimated 40 hours. A planning committee will coordinate communication, proposals, and speakers. SummaryofActivities
Code4lib NW was held on June 7, 2010. The conference had 65 attendees. Registration filled up within days and a wait list was required. At the conference, it was decided not to combine with OnlineNW or another conference but to remain a discrete event. In the post conference survey, 100% of respondents rated the conference experience as either Excellent or Good (66% Excellent). The conference was self‐supported with registration fees from participants. By holding the event at the UO White Stag building in Portland, we are able to keep registration fees down to $50. In order to broaden the community of participants and give more leadership opportunities within the program, a committee was formed to plan the 2011 conference. Members of the committee include: Doug Erickson from Seattle University, Mike Flakus from Portland State University, and Wade Guidry from Seattle Pacific University. Next year’s conference will be held on June 13, 2011 at White Stag in Portland. 7
2.PreservationStorage
Recommendations
●
●
●
●
Expand availability of cloud servers as systems testing and production environments Expand Offline Storage project to include more institutions Continue investigation of preservation storage solutions including: California Digital Library Microservices, iRODS, DuraCloud, etc. Complete participation in DuraCloud pilot for further review of cloud storage systems and preservation systems. Discontinue offering cloud storage as a product. If demand later emerges for this service, it can be re‐evaluated. SummaryofActivities
CloudStorage
A cloud storage proposal was set up using Amazon EC2 (appendix A), and advertised through email to the general Alliance announce list, in an Digital Services Team update, and through an announcement at code4lib NW. No institutions have ordered online cloud storage. As of this writing, it appears that staff perceive this service would be useful to other institutions, but not their own. However, UO, PSU, EOU, and WSU are using cloud servers ordered through the Alliance for systems. (See appendix B.) This service has proved beneficial for members as a quick solution to testing and running systems. In addition, as an opt‐in service, it provides a revenue stream to the Digital Services Program. DuraCloudPilot
DuraCloud is a new service from DuraSpace focused on providing preservation support services and access services. The DST Preservation/Storage subgroup submitted a proposal to participate in the DuraCloud pilot originally scheduled April‐July 2010. Our general interest was to evaluate preservation tools and storage methods, but we were interested in participating in the pilot specifically so we could explore efficient ways to identify and correct corrupt or outdated files, provide collective access to collections which must currently be accessed individually, and provide convenient for large collections including streaming content. The DST chair and the preservation storage sub‐group have participated in several calls to discuss the specific use cases we wish to test as well as webinars to impart knowledge necessary to participate in the project. The pilot was finally available in late September, and the group has tested some basic functionality of DuraCloud, including using the pilot synchronization tools to move large amounts of data to multiple storage facilities in the cloud. The next steps will to be to test some of the automated tools which address the use cases most relevant to the Alliance. 8
CurateCamp
Karen Estlund and Kyle Banerjee attended Penn State and California Digital Library’s inaugural Curate Camp to learn about an approach to digital curation based on combining small, interoperable, and self contained services known as “microservices” rather than relying on monolithic software packages designed to deliver all needed functionality. The expected benefit is that microservices are easier to maintain, deploy, and replace when technology cycles occur. Karen and Kyle both found the microservices model compelling but immature. It is more flexible, adaptable to technology changes, and is more easily maintained using commodity skills and tools. Such an approach will allow the Information Technology Manager to spend less time and resources maintaining services at a higher level. The recommendations of the DST IR/Hosting subgroup could also be easily combined with this model as a preservation service. We recommend continued investigation. OfflineStorage
In late July, PSU, OSU, and UO started participating in a pilot project to exchange back‐up tapes. (See appendix C.) Appropriate cases were ordered and procedures were developed and posted on the Alliance website at http://www.orbiscascade.org/index/cases‐for‐backup‐media. Nancy Nathanson provided support in creating documentation and procedures. The process seems to run smoothly, and the project may be expanded to accommodate more institutions. An official assessment of the program will be conducted in March 2011 by the Digital Services Program Manager. 9
3.PreservationTrainingandPolicy
Recommendations
●
●
●
Create a Digital Preservation Service within the Digital Services Program to create a system, workflow, and policy to ensure long‐term access to digital objects and work toward a trusted digital repository (See Appendix D.) Charge the Digital Services Program to complete the following specific activities in 2011: ○ Assemble a Digital Preservation Service Planning Team, composed of individuals from the Alliance and NWDA with key skills (See full planning document for details) ‐ January 2011 ○ Assemble a small group of institutions to form the pilot group for the program ‐ June 2011 ○ Conduct a risk assessment of pilot group’s materials and an analysis of the skills and resources needed ‐ Summer/Fall 2011 ○ Articulate the skills and resources by the end of the calendar year ‐ October 2011 Coordinate preservation training concentrating on new and emerging areas of digital preservation led by the Digital Services Program Manager PolicySummaryofActivities
●
●
●
●
●
Analysis of the existing standards for preservation policy training Selection of TRAC (Trustworthy Archives Audit and Certification Checklist) model to pursue preservation planning and completion of basic‐level administrative TRAC audit (See Appendix E.) Survey of member institutions to identify legal, mission, and other mandates for digital preservation and long‐term access to digital content, which gives a strong indication of the likelihood that a digital preservation program would be supportable. The results of this survey show that one third of Alliance and NWDA members have administrative mandates that support participation in a digital preservation service. Completion of a preservation program planning document associated with the Lyrasis workshop (See Appendix D.) Analysis of file formats most frequently archived at member institutions TrainingSummaryofActivities
●
●
Coordinated two digital preservation basics webinars (Aug. 10 and Sept. 14) by Liz Bishoff, of the Bishoff Group, for administrators, archives and special collections personnel, and technical staff. The goal of the webinars was to provide an overview of what digital preservation is and isn’t and to give Alliance and NWDA members a common vocabulary and understanding so that we can knowledgeably discuss the potential for a digital preservation program at the Alliance. Thirty‐
three individuals attended the webinars. Jodi Allison Bunnell and Michael Klein (OSU) attended the Staying on TRAC: Digital Preservation for Digital Collaboratives workshop offered by Lyrasis. This NEH‐sponsored workshop is designed to develop the capacity of digital collaboratives and their members to assume responsibility for long‐term accessibility of digital collections under their stewardship and to expand the number 10
of collaborative organizations capable of assuming responsibility for long‐ term accessibility of their digital collections. With pre‐workshop webinars, two days of in‐person workshops, follow‐
up webinars, and the possibility of being chosen for personal consultation with workshop faculty, this workshop has advanced the work of this subgroup by offering expertise and structure for its work. Participation in this initiative will continue the rest of 2010 and into 2011 led by Jodi. 11
4.HostingSystems/IR
Recommendation
To create the most flexible solution, the Alliance should host a Fedora Commons instance with one or more interfaces such as: Islandora or Hydra/Blacklight. (Fedora does not have a native access interface.) Streaming support should also be investigated for viability during the pilot phase. The new IT Manager would spend approx. 3 months to set‐up and test system before production with two pilot institutions. Ongoing systems support to be provided with .25 FTE of the IT Manager and .25 FTE of the Digital Services Program Manager for training and support. See also Findings of Digital Asset Management Systems Evaluation (Appendix G.) for comparison and recommendations between other systems, which may be appropriate depending on individual organization needs. SummaryofActivities
●
●
●
●
●
●
●
●
●
●
Reviewed available systems Created initial criteria for review Contacted current users of systems for feedback Investigated collaborating with other consortia ○ Colorado Alliance ADR (Alliance Digital Repository) ○ LASR (Liberal Arts Scholarly Repository) Contacted vendors for consortial pricing information on various repository platforms including (Simple DL and CONTENTdm) Decided to split into different categories (Traditional Institutional Repository, Multimedia, Journal Systems, and Power Solutions) based on very different strengths of systems and wide array of member needs and non standard usage of the term IR Narrowed down systems per category Communicated criteria and list of systems to wider DST and other self‐identified interested individuals from Alliance institutions for review and feedback Installed and tested systems; set up vendor accounts for demos Conducted final review of systems for recommendations 12
5.Scanning/ReformattingRecommendations
Recommendation
The Digital Services Team recommends the Digital Program Manager continue to update public list of vendors and act as a liaison for member institutions with concrete needs to help identify potential partners with similar needs for cost savings. SummaryofActivities
We contacted several vendors to obtain pricing and possible cost saving for collaborative scanning/reformatting. We also contacted several institutions associated with DST members about projects ready for scanning and reformatting, but were unable to obtain concrete cases or numbers of items. Due to the nature of vendor needs and unestablished concrete use cases, we are unable to confirm any pricing models with vendors. To create a useful mechanism for scanning/reformatting needs in the Alliance, we compiled a list of recommended vendors by Alliance institutions. We outlined a process on the Digital Services Team website whereby if an institution has a concrete need with funds in hand, they can contact the Alliance and we can act as an intermediary to identify similar projects in the Alliance and negotiate favorable pricing. This same site allows libraries to recommend vendors. https://spreadsheets.google.com/ccc?key=0AqNE9BXUPDl9dGRkcVRxTXBVUXZiLXRvd0ZsdjN2bmc&hl=e
n#gid=0 13
6.Cross‐SearchUtility(formerlyDiscoveryPrototype)
Recommendation
The Digital Services Team recommends that the cross‐search utility (previously referred to as the discovery prototype) initiative move forward to pursue an implementation grant from the Institute for Museum and Library Services under the leadership of the Northwest Digital Archives (NWDA) program. This initiative is of great interest to the NWDA membership and has vital importance for the NWDA program's continued sustainability. The NWDA steering team endorses this recommendation and is willing to lead this project and incorporate any viable outputs into its current services and systems. The IMLS National Leadership grant, due February 1, 2011, if awarded, would allow work to begin on the cross‐search utility as early as October 2011 and can be for between $50,000 and $1,000,000 for up to three years. The IMLS application would be in the Demonstration category, which would allow us to build a replicable technical solution and explore how to sustain it without being obligated to do so. This would give the NWDA program and the Alliance an opportunity to solve an important problem without taking on the inherent risk of sustaining a system in the long term. This direction will allow this important initiative to move forward at a pace that will satisfy the needs and expectations of NWDA and Alliance members. SummaryofActivities
●
●
●
●
●
●
Evaluation of similar or other EAD and Special Collections/Archives Projects; consultation with key individuals in the field to ask for their assessment of the utility’s importance and uniqueness Contacted the University of Illinois‐Urbana Champaign and Texas Archival Resources Online/Texas Digital Library to explore partnerships Consulted with program officers from granting agencies on viability of project grant funding Re‐configured explanatory documents of system Implemented local instance of applying conceptual issues at Willamette University Surveyed Alliance and NWDA membership for interest in pursuing further development of Cross Search Utility ‐ Out of 48 institutions, 26 responded to the survey. 64% indicated they were interested and 16% very interested, for a total of 80% of respondents. Only 5 were either neutral or disinterested, and no one was very disinterested. Clearly this initiative has sufficient support to make it worthwhile to move forward. 14
7.FutureDirections
Recommendations
DigitalServicesDay
In order to help build community, communicate within the Alliance, and address needs of the users of digital services, the Digital Services Team recommends a “Digital Services Day” to present the work of the team. We recommend this meeting be help in Mid/Late January 2011 at the UO White Stag building in Portland and that it be broadcast so that persons unable to attend are able to understand the work of this team. OrganizationalStructurewithinAlliance
A Digital Services Steering Team should be created, modeled after the NWDA steering team with strong Alliance staff support. This group would help set priorities, convene working groups, and work with the Digital Services Program Manager and Alliance Board on strategic directions. This group should be small (no more than four or five people). There should, in turn, be small working groups associated with specific activities in Digital Services. The chairs of any small groups should be on the Digital Services Steering Team to facilitate communication and coordination. The Digital Services Steering Team is also in charge of widely communicating with Alliance and NWDA members and receiving feedback on issues. ProcesstoReviewNewIdeas/Strategies
The Digital Services Team recommends that a process be created to recommend new ideas through the Digital Services Steering Team. Due to the length of our current processes, institutions are seeking independent rather than collaborative solutions which could benefit the whole. Smaller collaborations among institutions may lead to new opportunities that may not be appropriate for all members, but are valuable collaborative solutions for groups within our larger collective. Decisions and programs would remain the purview of Council; however, investigations and explorations would be conducted more efficiently within the structure of the Digital Services Steering Team. 15
Appendices
AppendixA.CloudStorageProposal
Purpose: To provide Alliance members with offsite storage for data that must be updated or retrieved more quickly and/or frequently than can be achieved with tape. This solution is appropriate for applications requiring quick replication or retrieval of data to/from a remote site. For example, it could be used to recover a file within seconds that had inadvertently been deleted from a major website that is maintained by many people. This service is intended to be a backup, but not a digital preservation solution. Features: ● Windows, MacOS, UNIX access via GUI or command line tools ● Preserves file permissions and modification times ● Can use rules to selectively copy/exclude files ● Pay only for what you need in 1GB increments with no upper limit ● Includes unlimited bandwidth and support This service does not include: ● File integrity checking or repair ● Format migration ● Additional backup of uploaded files ‐‐ this copy is the backup Cost: 0.80/mo per GB plus $15.00 per invoice. By paying in advance, institutions can avoid the need for monthly invoicing. Features available at additional cost: ● Daily reports outlining added, deleted, and changed files (add 0.10/mo per GB) Services under development: ● File monitoring for specified formats to identify resources that might need migration ● Batch migration services 16
AppendixB.CloudBasedSandboxService(Pilot)
●
●
●
●
Do you have difficulty getting computing resources for technology experiments and short term projects? Do you need more flexibility than your current IT environment allows? Do business rules make it difficult for your office to work with cloud providers? Do you need a server today? If you answered "yes" to any of the above questions, consider participating in the Alliance's Cloud Based Sandbox Service (CBSS) Pilot. Getting started is easy ‐‐ all you need to do is contact Kyle Banerjee at 503‐999‐9787/[email protected] and you can be running the same day. No knowledge of cloud services is required, though a short free initial consultation is recommended to help you determine if CBSS is a good fit for your project. CBSS comes with predictable costs, flexibility, support, and invoicing. You can try much more ambitious projects ‐‐ if something fails, you can have your server rebuilt at no charge in just a few minutes. Includes: ●
●
●
●
●
●
●
●
●
●
●
Linux 32 bit single compute core instance with 1.7 GB RAM on Amazon cloud Basic setup including installation of common programs such as Apache and MySQL Firewall to limit access by port, IP, and/or IP range Fixed public IP 10 GB elastic block storage 20 GB inbound/outbound data transfer No additional limits on system use/configuration except SMTP traffic is throttled One time imaging of instance for use by institution Phone/email support for connecting to instance Recovery from imaged instance or reimaging based on public instance Single invoice This service does not include: ●
●
Backup/recovery services aside from one time image creation or recovery from images Technical consulting for specific software packages Cost: ●
●
●
$550 for 3 months. $750 for 6 months. $1200 for 12 months The pilot period will run from July 2010 to June 2011. Pilot is only available to Alliance member libraries but we'll consider the potential for offering this to non‐members as well. 17
Features available at additional cost: ●
●
●
●
●
Monthly invoicing Regular snapshot creation Regular image creation More disk space More bandwidth ●
●
●
●
More computing power Automated monitoring Load balancing Technical consulting 18
AppendixC.DistributedOfflineStoragePilot
(OSU, PSU, and UO) Purpose: To provide offsite storage for large amounts of data that cannot be efficiently stored online. This proposal allows for unlimited storage. It should be regarded as a very cost effective backup mechanism. It is not intended to be a digital preservation solution. Levels of risk addressed: ●
●
●
Physical loss (e.g., hardware failure, natural disasters)* Data loss (e.g., bit rot) Loss of context (e.g., loss of metadata) ‐ Item / collection level data can be stored with items * Locked cases will be transported via the Courier service and stored in areas of the library that are regarded as "safe" but not necessarily "high security." The risk of loss will be comparable to other materials shipped through the Courier and stored at such sites. However, the presence of multiple copies makes it extremely unlikely that all copies will be lost This service does not include: ●
●
●
Data integrity checking beyond what is done when media are written Encryption beyond what is already present in the data Format migration Cost: ●
●
●
●
Initial expense: Pelican cases for transporting backup media. Institutions in the pilot are recommended to purchase three Pelican 1490 cases http://www.pelican.com/cases_detail_specs.php?Case=1490 which typically cost $100‐$130 each depending on source Ongoing expenses: None beyond occasional case replacement (cases should last many years) and staff time sending/receiving cases Cases sent through the Courier must have the proper labels affixed as required by the Courier. It is recommended that a window be affixed to the case that can contain a label that is flipped as it moves back and forth between two institutions. When an institution receives a case, it should return an existing case it already has for that institution. For example, PSU receives a case from OSU. Staff check their storage area and see they already have an OSU case. That case is sent back to OSU, and the case that was just received is put in the storage area 19
Proceduresforpilot:
SHIPPING SIDE ○ Place tapes, hard drives, etc that you want to send in a case ○ Tape Summit label to outside of case with clear packing tape. Consider using AIGNER Bin Buddy Self‐Adhesive Label Holders or equivalent to make it unnecessary to tape labels ○ Leave with Courier materials RECEIVING a. Inform staff to expect these cases b. When case comes, place in designated area. The designated area can be any area that would be considered safe enough Summit materials. It can be in a processing area, circulation, systems, or anywhere deemed appropriate. c. If a second case comes from an institution that the receiving library is already holding a case for, the older case should be shipped back after attaching a Summit label. VERY IMPORTANT: Staff shouldn't just send any case back to the institution ‐‐ it must be the one from that institution RECOVERY a. Inform staff of location/purpose of cases b. Staff receive call or email from institution wishing to recover data c. Staff ship case back using procedures above Procedures for individual institutions: UO: First case goes to OSU, Second to PSU, then alternate between the two PSU: First case goes to UO, Second to OSU, then alternate between the two OSU: First case goes to PSU, Second to UO, then alternate between the two 20
AppendixD.DigitalPreservationServicesPlanningProposal
ExecutiveSummary
This document provides the basic background to develop a Digital Preservation service at the Alliance under the auspices of the Digital Services Program. Digital preservation is consistent with the mission of the Alliance and is supported by strong or moderate mandates at approximately one‐third of Alliance or Northwest Digital Archives (NWDA) institutions. Almost no Alliance or NWDA members have identified other options to solve their need to address digital preservation, and interest in having the Alliance provide this program remains high. Recommendations
●
●
●
Create a Digital Preservation Service within the Digital Services Program to create a system, workflow, and policy to ensure long‐term access to digital objects and work toward a trusted digital repository Complete the following specific activities in 2011: ○ Assemble a Digital Preservation Service Planning Team, composed of individuals from the Alliance and NWDA with key skills ‐ January 2011 ○ Assemble a small group of institutions to form the pilot group for the program ‐ June 2011 ○ Conduct a risk assessment of pilot group’s materials and an analysis of the skills and resources needed ‐ Summer/Fall 2011 ○ Articulate the skills and resources by the end of the calendar year ‐ October 2011 Coordinate preservation training concentrating on new and emerging areas of digital preservation led by the Digital Services Program Manager Background
During the 2007‐2009 needs assessment process for digital services at the Alliance, 76% of the Alliance and NWDA members expressed high interest in collaborative digital preservation solutions. For some institutions, this is because they are charged by statute to ensure preservation of born‐digital public records (e.g. college/university archives, public archives). For other institutions, interest in digital preservation comes less from a definite mandate than a general notion that ensuring the long‐term preservation of born‐digital materials and reformatted analog materials is an important mission for an archival repository. Accordingly, the 2009 Digital Program Working Group recommended that the Alliance Create a Digital Preservation Working Group or Task Force to investigate whether or not the Alliance might take steps toward becoming a Trusted Digital Repository or determine what activities it might support in order for member institutions to pursue that goal. The Digital Services Team was appointed in early 2010 to pursue, among other things, the policy and other frameworks necessary to create elements of digital services that the Alliance had found desirable. These elements include digital preservation, hosting digital content for access, cross‐search (discovery 21
prototype), and scanning and reformatting. The DST, in turn, is divided into four sub‐groups: Preservation Policy and Training, Hosting/IR Hosting, Preservation/Storage, and Cross‐Search. The Preservation Policy and Training subgroup has worked in parallel with the Preservation/Storage subgroup in 2010 to create a service proposal for a Digital Preservation Service with policies for the consortium. The consortium would work with participating institutions to create appropriate best practices and workflows to create a viable and useful service. The purpose of a repository would be to support sustainable access to deposited born‐digital content from participating members. Of paramount importance is the ideal that members can create a more viable and lower‐cost service working together than attempting a preservation repository as individual institutions. Purpose The Digital Preservation Service would provide a system, workflow, and training to provide reliable, long‐term access to managed digital resources to its designated community, now and in the future. The system would adhere to and remain compliant with changing standards and best practices. The service will provide guidance on copyright, software retention, file migration best practices, and tools for those who also wish to manage their own digital preservation. Designated Community The designated community (aka user community) for the Digital Preservation Service will be the institutions that are the members of the service and their staff. Those institutions and staff in turn act as proxies for their stakeholder groups. They must be able to access materials that they submit to a Digital Preservation Service with a minimum of technical work and knowledge on their part. Mandate/Mission The Orbis Cascade Alliance is a consortium of academic institutions in Oregon and Washington. The mission of the Alliance is to strengthen member libraries through collaboration in order to support the work of our students, faculty, staff, and researchers. Alliance members join together to enhance our services, share our information resources and expertise, enrich and preserve our collections, and develop library staff to meet the challenges of a rapidly changing information environment. (http://orbiscascade.org/index/mission‐statement) During the 2007‐2009 needs assessment process for digital services, 76% of our membership (Alliance and NWDA members) expressed high interest in collaborative digital preservation solutions. No one said that they already had a satisfactory solution for their digital preservation needs. In a follow‐up survey in September 2010, we asked about the materials for which institutions were responsible, what types of legal and other mandates they had to materials access and preservation, and whether they were in conversation with any other entities about digital preservation services. Twenty‐
six of the 48 Alliance or NWDA institutions responded. ● Twenty‐three institutions (88%) were responsible for born‐digital resources or digital versions of resources that they had created or acquired as part of archival and manuscript collections; 77% 22
●
●
●
●
were responsible for administrative, scholarly, and other digital resources created by their organizations; and 19% were responsible for commercially available digital resources that they had purchased. Thirteen institutions (54%) said that their mission stated a commitment to long‐term preservation and accessibility of materials. In comments where respondents quoted from these mission statements, they ranged from very strong and specific (in the organization’s mission or current strategic plan) to much more loose and reflecting an individual’s commitment rather than that of an institution (“we have this responsibility as a profession”), with a preponderance of the former. Eight institutions (33%) stated that there are statutory or regulatory documents that committed their institution to long‐term preservation and access. The comments on these answers showed that this is a strong area of mandate, with this responsibility stated in state or municipal codes of Washington, Oregon, Montana, and the City of Seattle. Ten institutions (43%) stated that they are the official and legally‐recognized repository for materials that must be preserved. This is another strong mandate; comments showed that this recognition extends responsibility for mostly records of higher education and municipal records. Sixteen institutions (67%) said that they have accepted donated materials for which they have made a specific, written, commitment to long‐term preservation. It does not appear that institutions have made other arrangements for digital preservation services since the last survey on this topic in 2007. Two NWDA members in Montana are using OCLC’s Digital Archive as part of the Montana Memory Project, but are very interested in other options. Although institutions had had conversations with other entities, including the Washington State Digital Archives, no one stated that they had made a commitment to another solution, and several said that they are eagerly awaiting a solution from the Alliance. Clearly, ten to fifteen individual institutions have strong missions, legal responsibilities, and other strong commitments to digital preservation to provide a strong underpinning for a consortium‐level digital preservation service. The commitment of Alliance and NWDA members to the solutions described here will be most clearly articulated when it comes time to vote to support a service and/or to join it, but it appears that there are sufficient mandates at institutions to make this a viable and important service for the Alliance to include in its program offerings. The mission of the (proposed) Digital Preservation Service is: ● To support the long‐term preservation of digital content (born digital and reformatted digital) from a wide variety of organizations by providing a system that functions as a dark archive; ● To support the goals of its members, including teaching, scholarship, legal requirements, and administration; ● To promote a robust and cooperative approach to digital preservation; ● To maintain minimal overhead so that the cooperative approach is not only the most effective, but the least costly; 23
●
●
●
●
To facilitate participation even with varying levels of knowledge of digital preservation at participating repositories; To ensure that digital materials are stored and maintained in formats and data structures that can be accessed in the future; To recommend best practices for digital preservation and assist participating and potential Digital Preservation Service members with developing local best practices and policies; To become the digital preservation solution on which our member institutions rely for all or some of their digital preservation needs. The authoritative mandate for this service arises from the mandates at participating institutions (detailed above) and is created through the memorandum of agreement each participating institution completes with the Alliance. Individual institutions have a mandate to preserve born‐digital materials for the long term, particularly public records, under state records laws and corresponding local fiscal and legal mandates. Other situations or authorities creating mandates for digital preservation include donor agreements signed on the acquisition of private collections and general institutional missions. Challenges include: ● The diversity of institutions involved and their corresponding needs; ● Separating born digital from digitized content and potentially other types of separations, and meeting the management needs of both; ● The technical problems that any preservation program confronts, including managing version control and migration issues; complex and dynamic objects; software and services required to access objects; ● Legal requirements on governance and deposit in accordance with state laws; ● Helping institutions make the transition from worrying about digital preservation to putting institutional resources toward solutions; ● The changing technological landscape; ● Garnering the financial support of a sufficient number of institutions to create an effective service. The materials with which the service is concerned are at‐risk digital materials for preservation, such as born digital content, content reformatted from analog originals, and/or mandated retention. The extent to which the membership is responsible for these materials is detailed above. Current Organizational Structure & Staffing Current staffing at the Alliance is as follows: ● Executive Director (1.0 FTE). Manages the Alliance and its staff. Professional librarian, background in management and library systems. ● Resource Sharing Program Manager (1.0 FTE). Position manages the Summit resource sharing system. Professional librarian, background in resource sharing and bibliographic management. 24
●
●
●
●
●
●
●
Digital Services Program Manager (1.0 FTE). Position manages the developing Digital Services program. Professional librarian, background in library systems and bibliographic management. Northwest Digital Archives Program Manager (0.5 FTE). Position manages the NWDA EAD program. Professional archivist, background in all aspects of archival management. Electronic Resources Program Manager (1.0 FTE). Position manages the Electronic Resources and Collection Development programs. Professional librarian, background in electronic resource purchasing, contract and price negotiation, and cooperative collection development. Training Coordinator (0.5 FTE). Position supports training for the Summit resource sharing system and provides documentation for that system. Professional with a background in database management. IT Manager (1.0 FTE) (currently vacant). Position will support current technical functions at the Alliance (Summit system, NWDA, cloud computing, content management system, website, administrative databases) and will evaluate and implement technologies associated with the Strategic Agenda. Business Manager (1.0 FTE). Position supports business functions at the Alliance, including financial oversight, contracts and purchasing, and management of the Courier program. Professional with a management and human resources education background. Administrative Assistant (1.0 FTE). Position supports all the administrative functions at the Alliance. Paraprofessional with organizational and support skills. Governance of the Alliance under its new incorporation is provided by an eight‐member Board of Directors (documented at http://orbiscascade.org/index/board), which carries out the active governance of the organization; and a governing Council, which has one voting member from each of the thirty‐six full members of the Alliance. The Alliance currently maintains a number of contractual relationships, including legal counsel, desktop support, Courier vendor, and many purchasing agreements for licensed electronic content. Full Alliance members and program members sign a Memorandum of Understanding with the Alliance that provides a framework for provision of services. The organization has a very adequate legal and financial infrastructure for these relationships that maintains a balance of specificity and generality. The Digital Services Program has a Digital Services Team associated with the Strategic Agenda and program development for the calendar year 2010. Unlike other programs at the Alliance (i.e. NWDA, Collection Development and Management), Digital Services does not currently have a Steering Team or Committee composed of program participants. Needed Organizational Structure & Staffing A Digital Preservation Service will naturally draw on the skills and time of its current governing board and staff. Primary staff to be involved may include the Digital Services Program Manager, the NWDA Program Manager, IT Manager, Business Manager, Administrative Assistant, and Executive Director. Personnel requirements to support a Digital Preservation Program will depend on a number of factors, 25
including program membership and the number and complexity of materials the program is called on to handle. Needed skills and expertise to support a Digital Preservation Service include: ● Deep familiarity and experience with archival best practices of all types, but particularly preservation; ● Deep familiarity and experience with preservation (digital, analog, or both); ● Systems administration; ● Familiarity with digital file formats and conversion tools; ● Legal, fiscal, and administrative expertise to guide needed program development. The Digital Preservation Service will also need organizational structure similar to other Alliance programs. To determine exact staffing levels needed that include current and additional staff FTE and skills that are present, absent, or need augmentation, the program will need to conduct a thorough gap analysis in the near future. An important aspect of Digital Preservation Service staffing, and one that will require regular and ongoing financial support, is an active professional development program for its staff. This will ensure that they are able to keep abreast of a rapidly changing technological and policy landscape to provide program members with a robust service. It is also entirely possible that the program will need to provide similar professional development and training for its program members so they that they will be able to participate knowledgeably in the program and its governance. Roles & Responsibilities The Alliance, its board of directors, its staff, and any relevant contracted services will provide this program as a service in exchange for fees from program participants. DP Program participants will maintain membership in good standing by complying with program requirements and guidelines, which will include providing appropriate metadata and complying with file format requirements. They support at their own expense any and all costs incurred by participating in the program, including but not limited to paying membership fees and travel to required meetings. They implement appropriate standards for addressing copyright and other issues related to contributed content in order to comply with local, state, federal, and international law, The program is a collaboration between members of the Alliance and other organizations that may join and support this program. In addition, during its founding and subsequent operations, the program will 26
be open to and will seek appropriate collaborating partners including, but not limited to, those that can advise on program development and those that provide technical infrastructure. Once the program is established, it will conduct a review of the MOU between the program and program participants every five years to ensure continued accuracy and relevance. Legal Permissions The program will need to establish its right for the digital repository to work with and potentially modify digital objects to keep them accessible over time. Written policies and agreements with depositors will specify and/or transfer particular rights to the DP Program and will allow it to take certain preservation actions and establish appropriate liability and legal exposure for the organization. These policies and agreements will need to be developed with the pilot members of the DP Program. Documentation The program will need to develop an initial set of administrative, policy, and technical documentation that will record the decisions made about the organizational and technological infrastructure. This will need to be maintained over time to document the DP Program’s activities and developments. Financial Sustainability Following general practices of the Alliance, program members will establish a funding model to include general funds and program funds. The program will follow the good business practices of the Alliance that have kept the organization financially healthy since its inception. The program will seek external funding to expand the scope and capabilities of program, but will depend on the support of its members for basic operations. Risk Assessment Risks to digital content are numerous and serious. Understanding and documenting them is a key underpinning to any digital preservation program. General risks to digital content may include technical, physical, organizational, socio‐cultural, legal, financial, political, contractual, and force majeur (i.e. fire, earthquake). The DP Program will undertake a risk assessment with its pilot members to understand and document the real and potential threats to the digital collections, producers, and consumers. This assessment will include an inventory of file formats, technology infrastructure, legal mandates, and staffing. Rights Management In all aspects of a DP Program, preservation of digital resources will include complying with the IP rights and/or other legal rights related to copying, storage modification and use of specific resources. 27
The agreement between the Alliance and DP Program members will clearly define the rights, responsibilities, and expectations of each party. They will need to be mildly more specific than other agreements that the Alliance maintains with its members in order to provide the needed underpinning of a trusted repository, but will strive to balance that need with the one to avoid excessive administrative overhead. Technical Assessment The technical infrastructure for the DP Program is to be determined pending approval for creation of the program, gathering an initial pilot group of institutions, and administrative and policy work. Preservation & Quality Control See the attached formats document for more information on formats and strategies. Preservation Metadata The DP Program will follow appropriate national standards and best practices that fit our collections as established by organizations including the Library of Congress, the National Archives and Records Administration, the Digital Library Federation, and others. Evaluation and Updating The DP Program will develop a process and timeline for regular review and updating of its policy and administrative documents. Consortial Activities In some cases, it will be more appropriate for another consortium or a vendor to provide long‐term preservation services for some formats and needs. In those cases, and if it is of mutual benefit to all parties involved, the DP Program will work cooperatively with that entity to facilitate that service. Moving Forward Pending approval for further activities toward the creation of a DP Program, we recommend that the following activities be carried out in 2011: 2. At the beginning of the year, assemble a Digital Preservation Service Planning Team that will be charged with creating a program ready to work with content from a pilot group of members by the end of the calendar year. It will be responsible for both administrative and policy work and the technical (hosting) infrastructure for the program. The DPSPT should consist of individuals with the following skills from the Alliance staff and Alliance and/or NWDA institutions: d. Technical expertise, including (but not limited to) systems administration, tools evaluation, and the ability to estimate processing and storage requirements; e. Familiarity with the principles of digital curation; f. Archives and archival principles; g. Preservation (analog, digital, or both); h. Metadata, including descriptive, administrative, and preservation; 28
i.
Business and financial; j. Legal; k. Administration and governance, including familiarity with the principles of TRAC. ●
●
●
●
●
By mid‐year, assemble a small group of institutions to be the pilot members in the program. Institutions should include a mix of institution types, sizes, and ability to contribute expertise, but all should be able to contribute something. As with other Alliance programs, a single representative of each institution will serve on a Digital Preservation Service Steering Team and work with the DPSPT. They should expect to have the program working with at least a small sample of their content by 2012. This pilot would proceed on a mix of no‐cost and cost for participation. Once the initial pilot group is assembled, work with the members of that group to conduct a risk assessment for their materials. Once the scope of those materials and needs are known, conduct a gap analysis of needed staff resources and technical infrastructure to address those needs. By the end of the year, be able to articulate the resources needed to develop a Digital Preservation Service. Preservation training should continue to be a focus of Digital Services activities and concentrate on new and emerging areas of digital preservation 29
AppendixE.TrustworthyArchivesAuditandCertification
Checklist:AdministrativeSection
The TRAC audit document is available at the CRL website, specifically http://www.crl.edu/sites/default/files/attachments/pages/trac_0.pdf. Documentation of the Alliance’s commitments and practices was taken from the Alliance’s public website, http://orbiscascade.org/index/index. It does not include documentation that is password‐
protected or otherwise not publicly available. Throughout this document, the TRAC elements are in boldface, documentation is in regular font, and commentary is in italics. This preliminary audit was completed by Jodi Allison‐Bunnell, NWDA Program Manager, on 2010 August 25. A1. Governance & organizational viability The Orbis Cascade Alliance is in the midst of an important transition that affects a great deal of the administrative documentation available to complete this audit. The Orbis Cascade Alliance and its predecessor organizations have served nonprofit college and university libraries in Oregon and Washington since the early 1990's. Originally organized as an unincorporated association, the Alliance's home and fiscal agent is the University of Oregon, with a MOU between UO and each member serving as the legal basis for the consortium. This membership MOU was approved by the Oregon Department of Justice in 1993 and a revised version of this document was approved by the Attorney General Office in Oregon and Washington and signed by all members in 2003. The Alliance has been an unincorporated association for more than 15 years and has added many members and programs in the intervening years. A small project with a budget of $100,000 has grown to become an internationally respected consortium with eight staff, 36 members, more than 200 non‐
members served in six states, and a budget of $5.7 million. Member institutions see this regional consortium as a key venue for their advancement and ability to provide first‐rate services to faculty, students and researchers. The members' ambition for their consortium is clearly evident in the organization's Strategic Agenda, featuring six broad initiatives under investigation by more than 40 staff at member libraries. The Alliance is a great success and is now working to become an Oregon nonprofit corporation in the summer of 2010. This move is the latest evolutionary step for an organization that has always carefully tended to its financial health and legal foundation while moving ahead with great ambition. Although 30
incorporation will bring some important changes, the services, strategic agenda, mission, and principal players remain the same. A1.1 Repository has a mission statement that reflects a commitment to the long‐term retention of, management of, and access to digital information. The mission of the Orbis Cascade Alliance is: The Orbis Cascade Alliance is a consortium of academic institutions in Oregon and Washington. The mission of the Alliance is to strengthen member libraries through collaboration in order to support the work of our students, faculty, staff, and researchers. Alliance members join together to enhance our services, share our information resources and expertise, enrich and preserve our collections, and develop library staff to meet the challenges of a rapidly changing information environment. (http://orbiscascade.org/index/mission‐statement) Additionally, the Alliance’s mission is described thus in the Memorandum of Understanding that full members sign (available at http://orbiscascade.org/index/cms‐filesystem‐
action/legal/final/memorandum_of_understanding_2010.07.doc): The mission of the Alliance is to strengthen member libraries through collaboration in order to support the work of students, faculty, staff, and researchers. Alliance members join together to ■Enhance services; ■Share information resources and expertise; ■Enrich and preserve collections; ■Develop library staff to meet the challenges of a rapidly changing information environment; ■Support other activities that further goals established by the Council of library directors. The Digital Services Program is described as follows: The Digital Services Program (DSP) provides coordination and support for online services delivered at the network level in accordance with goals outlined in the Alliance's strategic agenda (see http://orbiscascade.org/index/strategic‐agenda). In addition, the DSP identifies best practices for provision of digital services and develops or secures the infrastructure necessary to support collection, analysis, and provision of digital information necessary as required by other Alliance programs. (From http://orbiscascade.org/index/digital‐services) Both statements reflect a commitment to collaboration and preservation of collections, from which a commitment to digital preservation may be derived. Digital preservation is completely consistent with the Alliance’s mission. A1.2 Repository has an appropriate, formal succession plan, contingency plans, and/or escrow arrangements in place in case the repository ceases to operate or the governing or funding institution substantially changes its scope. 31
The articles of incorporation for the Alliance http://orbiscascade.org/index/cms‐filesystem‐
action/legal/final/articles_of_incorporation_2010.06_signed.pdf) state: Article V: Upon the dissolution or final liquidation, after the payment or provision for payment of all of the liabilities of the corporation, the remaining assets of the corporation shall be distributed to such organization or organizations that are then described in IRC Sections 501(c)(3), 170(c)(2), 2055(a)(2), and 2522(a)(2) and/or to the United States or any state for exclusively public purposes as the board of directors shall determine. A digital preservation program would need to identify a more specific succession plan than currently exists in these articles of incorporation. It should include a specific successor or successors who would continue to steward the digital assets, the option for member(s) to adopt, or specify that the assets are returned to their depositing organizations. A2. Organizational structure & staffing A2.1 Repository has identified and established the duties that it needs to perform and has appointed staff with adequate skills and experience to fulfill these duties. Current staffing at the Alliance is as follows: Executive Director (1.0 FTE). Manages the Alliance and its staff. Professional librarian, background in management and library systems. Resource Sharing Program Manager (1.0 FTE). Position manages the Summit resource sharing system. Professional librarian, background in resource sharing and bibliographic management. Digital Services Program Manager (1.0 FTE). Position manages the developing Digital Services program. Professional librarian, background in library systems and bibliographic management. Northwest Digital Archives Program Manager (0.5 FTE). Position manages the NWDA EAD program. Professional archivist, background in all aspects of archival management. Electronic Resources Program Manager (1.0 FTE). Position manages the Electronic Resources and Collection Development programs. Professional librarian, background in electronic resource purchasing, contract and price negotiation, and cooperative collection development. Training Coordinator (0.5 FTE). Position supports training for the Summit resource sharing system and provides documentation for that system. Professional with a background in database management. IT Manager (1.0 FTE) (currently vacant). Position will support current technical functions at the Alliance (Summit system, NWDA, cloud computing, content management system, website, administrative databases) and will evaluate and implement technologies associated with the Strategic Agenda. Business Manager (1.0 FTE). Position supports business functions at the Alliance, including financial oversight, contracts and purchasing, and management of the Courier program. Professional with a management and human resources education background. 32
Administrative Assistant (1.0 FTE). Position supports all the administrative functions at the Alliance. Paraprofessional with organizational and support skills. These positions are adequate for current functions. The IT Manager position is designed to allow the DSPM and RSPM to focus on less technical duties and is designed to have room for expansion under the Strategic Agenda, of which digital preservation may be a part. The DSPM would be the most likely person to provide major support for a digital preservation (hereafter DP) program, and there may be capacity in that position for that function. The NWDA PM position could add needed archival expertise to a DP program. The full needs analysis for duties of staff associated with a DP program has yet to be completed. Depending on the size and complexity of the DP program, additional staff may be needed to support a program. Governance of the Alliance under its new incorporation is provided by an eight‐member Board of Directors (documented at http://orbiscascade.org/index/board), which carries out the active governance of the organization, and a governing Council, which has one voting member from each of the thirty‐six full members of the Alliance. The Digital Services Program currently has a Digital Services Team associated with the Strategic Agenda and program development. Unlike other programs at the Alliance (i.e. NWDA, Collection Development and Management), Digital Services does not currently have a Steering Team composed of program participants. The Alliance also contracts with appropriate entities for desktop computer support, legal counsel, Courier services, and other services. Pending approval for DP program creation, the program will need a committee or steering team consisting of potential or committed program members, consistent with management of other Alliance programs, to flesh out and implement the program. A2.2 Repository has the appropriate number of staff to support all functions and services. Staff and current areas of responsibility are detailed above. Current staff are adequate for current responsibilities. Addition of a new program may necessitate additional staffing or a different configuration of skills. A2.3 Repository has an active professional development program in place that provides staff with skills and expertise development opportunities. Documentation on this point not available; supplied through personal knowledge of staff member. 33
All Alliance staff are actively involved in professional development relevant to their field(s) and program areas. The organization provides funding in its annual budget to support those activities. Documentation of these activities could be found in the annual evaluation documents associated with each position. As part of these evaluations, each staff member sets goals for all areas of work, including professional development, with the Executive Director and evaluates the extent to which those goals were fulfilled at the end of each calendar year. A3. Procedural accountability & policy framework A3.1 Repository has defined its designated community(ies) and associated knowledge base(s) and has publicly accessible definitions and policies in place to dictate how its preservation service requirements will be met. The designated community and knowledge base is found in the member institutions of the Alliance. Policies and definitions for preservation service requirements will be defined as the Digital Services Program matures and if future work on digital preservation is approved by the Board of Directors and Council in late 2010. Not currently in place. The Digital Services Team will propose a designated community for a DP program in its October 2010 recommendations to the Board of Directors. If approved, the definitions and policies will be created and made publicly accessible. A3.2 Repository has procedures and policies in place, and mechanisms for their review, update, and development as the repository grows and as technology and community practice evolve. Not currently in place. Procedures and policies have yet to be created pending approval for the creation of a DP program. If approved, we recommend an annual meeting of the community to review these issues. A3.3 Repository maintains written policies that specify the nature of any legal permissions required to preserve digital content over time, and repository can demonstrate that these permissions have been acquired when needed. Not currently in place. As part of its work before making its proposal to the Board of Directors, the DST will determine legal and other mandates for digital preservation at potential member repositories and document them. These requirements will be articulated in MOU’s with participating institutions. A3.4 Repository is committed to formal, periodic review and assessment to ensure 34
responsiveness to technological developments and evolving requirements. Not currently in place. Pending approval for DP program creation, this commitment would need to be articulated. If approved, we recommend an annual meeting of the community to review these issues. A3.5 Repository has policies and procedures to ensure that feedback from producers and users is sought and addressed over time. Assessment of programs and services at the Alliance occurs at the program level and is initiated by the Program Managers. Pending approval for DP program creation, these policies and procedures would need to be created. If approved, we recommend an annual meeting of the community to receive feedback. A3.6 Repository has a documented history of the changes to its operations, procedures, software, and hardware that, where appropriate, is linked to relevant preservation strategies and describes potential effects on preserving digital content. Not currently in place. Pending approval for DP program creation, this documentation and the continued commitment to its production would need to be created and maintained by the IT Manager and Digital Services Program Manager. A3.7 Repository commits to transparency and accountability in all actions supporting the operation and management of the repository, especially those that affect the preservation of digital content over time. This commitment is not currently a stated goal or documented practice of the Alliance; however, it is part of its practices: all documentation that is not specifically confidential (i.e. vendor bids, confidential correspondence) is openly available. Minutes are kept of all governing and working groups and made available on the Alliance website. Pending approval for DP program creation, this commitment would be articulated in MOU’s with participating institutions. A3.8 Repository commits to defining, collecting, tracking, and providing, on demand, its information integrity measurements. Not currently in place. 35
Pending approval for DP program creation, this commitment would need to be specifically stated. A3.9 Repository commits to a regular schedule of self‐assessment and certification and, if certified, commits to notifying certifying bodies of operational changes that will change or nullify its certification status. Not currently in place. Pending approval for DP program creation, this commitment would need to be specifically stated. A4. Financial sustainability A4.1 Repository has short‐ and long‐term business planning processes in place to sustain the repository over time. Council has worked on business planning over the years that the Alliance has been in operation, but those documents are not immediately evident on the organization’s website. Pending approval for creation of a DP program, and as part of program creation, the program would need to create a business plan that includes potential membership, revenues, and costs, and how those tie into the strategic agenda and the Alliance’s long‐term business planning processes. A4.2 Repository has in place processes to review and adjust business plans at least annually. Not currently in place. Pending approval for DP creation, and as part of program creation, these processes would need to be created. A4.3 Repository’s financial practices and procedures are transparent, compliant with relevant accounting standards and practices, and audited by third parties in accordance with territorial legal requirements. Not currently in place, but in the process of being created These requirements are specified as part of articles of incorporation. Pending approval for DP creation, and as part of program creation, these commitments would need to be stated. A4.4 Repository has ongoing commitment to analyze and report on risk, benefit, investment, and expenditure (including assets, licenses, and liabilities). 36
Not currently in place, but in the process of being created as part of incorporation. Participating institutions would need to do a self‐assessment in these areas, as well, since risk is highly variable among member institutions. A4.5 Repository commits to monitoring for and bridging gaps in funding. Not currently in place, but in the process of being created as part of incorporation. Pending approval for DP creation, and as part of program creation, these commitments would need to be stated. Participating institutions would also need to anticipate these needs. A5. Contracts, licenses, & liabilities A5.1 If repository manages, preserves, and/or provides access to digital materials on behalf of another organization, it has and maintains appropriate contracts or deposit agreements. The Alliance currently maintains two types of agreements with its general and program members: a general membership agreement and agreements specific to program members (i.e. NWDA, Courier, ER) who may not be general Alliance members. The latter is structured as an MOU (see example at http://orbiscascade.org/index/cms‐filesystem‐action?file=nwda/nwdamembership%20agreement‐
fromdoj‐general.doc). These agreements are quite general to keep administrative overhead to a manageable level. and avoid lengthy Department of Justice review for each agreement. Pending approval for DP program creation, these contracts or deposit agreements will need to be created, both with program members and any third‐party providers with which the Alliance may contract for DP services or support. Current practices for agreements may not have the degree of specificity required for a DP program, and this practice would have to be assessed. A5.2 Repository contracts or deposit agreements must specify and transfer all necessary preservation rights, and those rights transferred must be documented. Not currently in place. Pending approval for DP program creation, these agreements and policy statements would need to be created for program members. A5.3 Repository has specified all appropriate aspects of acquisition, maintenance, access, and withdrawal in written agreements with depositors and other relevant parties. Not currently in place. 37
Pending approval for DP program creation, these agreements and policy statements would need to be created for program members. A5.4 Repository tracks and manages intellectual property rights and restrictions on use of repository content as required by deposit agreement, contract, or license. Not currently in place. Pending approval for DP program creation, these agreements and policy statements would need to be created for program members. A5.5 If repository ingests digital content with unclear ownership/rights, policies are in place to address liability and challenges to those rights. Not currently in place. Pending approval for DP program creation, these agreements and policy statements would need to be created for program members. [Subsequent sections address infrastructure TBD if DP program is approved.] B. Digital Object Management B1. Ingest: acquisition of content B1.1 Repository identifies properties it will preserve for digital objects. Not currently in place. Pending approval for creation of a DP program, the program would need to identify these properties and document them in a program mission statement, submission agreements, workflow and policy documents, definition of properties, written processing procedures, and other documentation. B1.2 Repository clearly specifies the information that needs to be associated with digital material at the time of its deposit (i.e., SIP). Not currently in place. Pending approval for creation of a DP program, the program would need to specify this information in transfer requirements/best practices and/or producer‐archive agreements. B1.3 Repository has mechanisms to authenticate the source of all materials. Not currently in place. 38
Pending approval for creation of a DP program, the program would need to specify this information in submission agreements, workflow documents, documentation of technological measures, and logs for procedures and authentication. B1.4 Repository’s ingest process verifies each submitted object (i.e., SIP) for completeness and correctness as specified in B1.2. Not currently in place. Pending approval for creation of a DP program, the program would need to specify this information through policy documents, system log files from ingest procedures, acquisitions registers of files received during ingest, documentation of procedures, and definition of completeness and correctness, incorporated in policy documents. B1.5 Repository obtains sufficient physical control over the digital objects to preserve them. Not currently in place. Pending approval for creation of a DP program, the program would need to specify this information through submission agreements, workflow documents, and system log files. B1.6 Repository provides producer/depositor with appropriate responses at predefined points during the ingest processes. Not currently in place. Pending approval for creation of a DP program, the program would need to specify this information through submission agreements, workflow documentation, and operating procedures. B1.7 Repository can demonstrate when preservation responsibility is formally accepted for the contents of the submitted data objects (i.e., SIPs). Not currently in place. Pending approval for creation of a DP program, the program would need to specify this information through submission agreements and confirmation receipts sent on receipt of each SIP. B1.8 Repository has contemporaneous records of actions and administration processes that are relevant to preservation (Ingest: content acquisition). Not currently in place. 39
Pending approval for creation of a DP program, the program would need to specify this through its workflow documentation and practices for creating and storing preservation metadata. B2. Ingest: creation of the archivable package B2.1 Repository has an identifiable, written definition for each AIP or class of information preserved by the repository. Not currently in place. The Digital Services Team is drafting a proposed list of formats that a DP program would support. Pending approval for creation of a DP program, the list of formats would need to be finalized with levels of support offered for each type of document. B2.2 Repository has a definition of each AIP (or class) that is adequate to fit long‐term preservation needs. Not currently in place. Pending approval for creation of a DP program, the list of formats would need to be fully defined in order to document the significant properties that must be preserved for each one. B2.3 Repository has a description of how AIPs are constructed from SIPs. Not currently in place. Pending approval for creation of a DP program, the program will need to create this workflow documentation that includes the relationship of the SIP to the AIP, standards or process against which any normalization occurs, and normalization outcome(s). B2.4 Repository can demonstrate that all submitted objects (i.e., SIPs) are either accepted as whole or part of an eventual archival object (i.e., AIP), or otherwise disposed of in a recorded fashion. Not currently in place. Pending approval for creation of a DP program, the program will need to generate this documentation through system processing files, disposal records, submission agreements, provenance tracking, and system log files. B2.5 Repository has and uses a naming convention that generates visible, persistent, unique identifiers for all archived objects (i.e., AIPs). 40
Not currently in place. Pending approval for creation of a DP program, the program will need to create documentation describing naming conventions and determine how to create physical evidence of its application through system logs. B2.6 If unique identifiers are associated with SIPs before ingest, the repository preserves the identifiers in a way that maintains a persistent association with the resultant archived object (e.g., AIP). Not currently in place. Pending approval for creation of a DP program, the program will need to create workflow documents that provide evidence of traceability. B2.7 Repository demonstrates that it has access to necessary tools and resources to establish authoritative semantic or technical context of the digital objects it contains (i.e., access to appropriate international Representation Information and format registries). Not currently in place. Pending approval for creation of a DP program, the program will need to have and budget for a subscription to an appropriate registry. B2.8 Repository records/registers Representation Information (including formats) ingested. Not currently in place. Pending approval for creation of a DP program, and in the absence of international standards for Representation Information, the program will need to create and maintain a local format registry. B2.9 Repository acquires preservation metadata (i.e., PDI) for its associated Content Information. Not currently in place. Pending approval for creation of a DP program, the program will need to create a local format registry for the preservation metadata and persistently link it to the relevant digital object(s). B2.10 Repository has a documented process for testing understandability of the information content and bringing the information content up to the agreed level of understandability. 41
Not currently in place. Pending approval for creation of a DP program, and once the designated community is fully defined, the program will need to create a process by which it can determine whether information content is understandable by those who need to use it, or if additional measures need to be taken. B2.11 Repository verifies each AIP for completeness and correctness at the point it is generated. Not currently in place. Pending approval for creation of a DP program, the program will need to create a process for this verification through checksums or other means, then put a procedure in place to regularly generate logs of the procedure. B2.12 Repository provides an independent mechanism for audit of the integrity of the repository collection/content. Not currently in place. Pending approval for creation of a DP program, the program will need to create a means for doing this. Documentation produced for practices B2.1‐B2.6 is partly adequate, together with submission agreements, logs of materials received, and logs of periodic checks. B2.13 Repository has contemporaneous records of actions and administration processes that are relevant to preservation (AIP creation). Not currently in place. Pending approval for creation of a DP program, the program will need to create a procedure for documenting these actions through documentation and preservation metadata. B3. Preservation planning B3.1 Repository has documented preservation strategies. Not currently in place. Pending approval for creation of a DP program, the program will need to develop documentation that identifies preservation issues (i.e. degradation of storage media, obsolescence of media drives, the obsolescence of formats, and accidental or intentional file corruption) and the program’s strategy for dealing with each one. 42
B3.2 Repository has mechanisms in place for monitoring and notification when Representation Information (including formats) approaches obsolescence or is no longer viable. Not currently in place. Pending approval for creation of a DP program, the program will need to determine its strategy for monitoring RI and format obsolescence issues and plan for associated costs. Possible strategies may include a subscription to a format registry service, subscription to a technology watch service, and/or staff time to monitor technological obsolescence issues. B3.3 Repository has mechanisms to change its preservation plans as a result of its monitoring activities. Not currently in place. Pending approval for creation of a DP program, the program will need to not only create its original preservation planning and policy documentation, but identify an interval or intervals at which its strategies will come under review for possible revision. B3.4 Repository can provide evidence of the effectiveness of its preservation planning. Not currently in place. Pending approval for creation of a DP program, and pending full determination of the program’s designated community, the program will need to create a procedure to test and document the effectiveness of its preservation planning. The program could plan for periodic review and proof of usability of randomly selected digital objects held within the system and create a demonstrable track record for retaining usable digital objects over time. B4. Archival storage & preservation/maintenance of AIPs B4.1 Repository employs documented preservation strategies. Not currently in place. Pending approval for creation of a DP program, the program will need to develop this documentation and create a workflow to generate appropriate preservation metadata. B4.2 Repository implements/responds to strategies for archival object (i.e., AIP) storage and migration. Not currently in place. 43
Pending approval for creation of a DP program, the program will have to create a format watch strategy (previously addressed), have in place a plan to capture appropriate preservation metadata, and be able to create and document preservation strategies for objects as required. B4.3 Repository preserves the Content Information of archival objects (i.e., AIPs). Not currently in place. Pending approval for creation of a DP program, policy documents will need to specify treatment of AIPs and whether they will ever be deleted/removed from the repository once they are migrated. Additionally, policy documents must address how the program will create a demonstrable chain between AIPs and specify how workflows will create this documentation. B4.4 Repository actively monitors integrity of archival objects (i.e., AIPs). Not currently in place. Pending approval for creation of a DP program, policy documents and workflows will allow the repository to create fixity information for AIPs, most likely through a checksum. These checksums must, in turn, be tied to AIPs but stored separately so that if malicious alteration of an AIP occurs, the individual will not be able to simultaneously modify the fixity information. B4.5 Repository has contemporaneous records of actions and administration processes that are relevant to preservation (Archival Storage). Not currently in place. Pending approval for creation of a DP program, the program will need to generate and maintain written documentation of decisions and/or actions taken and will need to create and store preservation metadata for the digital objects under its care. B5. Information management B5.1 Repository articulates minimum metadata requirements to enable the designated community(ies) to discover and identify material of interest. Not currently in place. The designated community is not yet formally approved but will be proposed to consist of the organizations that join the program rather than their end users. Pending approval for creation of a DP program, the program will need to consult with the designated community to create use cases for common types of requests. This, in turn, will shape the minimum requirements for descriptive metadata. 44
B5.2 Repository captures or creates minimum descriptive metadata and ensures that it is associated with the archived object (i.e., AIP). Not currently in place. Pending approval for creation of a DP program, the program will need to articulate how it will capture descriptive metadata: by requiring producers to provide it, or supplying some metadata during ingest. The program will need to specify this through depositor agreements, system documentation, metadata policy documentation, workflow documentation, and a statement that describes what entity is responsible for capture/creation of minimum descriptive metadata and what happens if it is not present. B5.3 Repository can demonstrate that referential integrity is created between all archived objects (i.e., AIPs) and associated descriptive information. Not currently in place. Pending approval for creation of a DP program, the program will need to specify ways in which practices will ensure that each AIP has some descriptive information and that all descriptive information points to an AIP. The program will create this through its system documentation and technical architecture and its workflow documentation. B5.4 Repository can demonstrate that referential integrity is maintained between all archived objects (i.e., AIPs) and associated descriptive information. Not currently in place. Pending approval for creation of a DP program, and related to point B5.3, program must have systems in place to demonstrate that referential integrity is maintained between AIPs and their descriptive metadata through system documentation, technical architecture, and workflow documentation. B6. Access management B6.1 Repository documents and communicates to its designated community(ies) what access and delivery options are available. Not currently in place. Pending approval for creation of a DP program, the program will need to create access policies that specify the designated community, what members of that community can ask for, when, how, and any associated costs. B6.2 Repository has implemented a policy for recording all access actions (includes requests, orders etc.) that meet the requirements of the repository and information 45
producers/depositors. Not currently in place. Pending approval for creation of a DP program, the program will need to create a means for recording access actions that complies with the needs of its depositors. It should record and report basic statistics (number of access requests served) and record other information as needed, including much more detailed information about access. The extent of this detail must be established in the program’s basic policies and documentation. B6.3 Repository ensures that agreements applicable to access conditions are adhered to. Not currently in place. Pending approval for creation of a DP program, the program will need to create access policies that specify the degree of openness versus limitations by user community and by AIP. The program should create a series of use cases that illustrate and shape these policies in close collaboration with program members. There should be recording systems in place that show whether access was authorized or denied so that appropriate access can be documented. B6.4 Repository has documented and implemented access policies (authorization rules, authentication requirements) consistent with deposit agreements for stored objects. Not currently in place. Pending approval for creation of a DP program, the program will need to create access policies that are consistent with submission or deposit agreements. The program will need to create user credentials for member institutions that may be an IP address limiter, username and password, or something more complex. The system will need to be able to validate access mechanisms and to document validation procedures. B6.5 Repository access management system fully implements access policy. Not currently in place. Any program must be able to demonstrate that access policies are implemented by producing logs of access requests and performing regular and explicit tests of some types of access. B6.6 Repository logs all access management failures, and staff review inappropriate “access denial” incidents. Not currently in place. 46
Pending approval for creation of a DP program, and assuming that the repository would house some restricted materials, the repository must have some automated mechanism to note unusual denials and use them to identify security threats or failures in the access management system. The system must generate access logs, use some automated monitoring tools and generate error messages, and policies should specify regular review of access denial logs. B6.7 Repository can demonstrate that the process that generates the requested digital object(s) (i.e., DIP) is completed in relation to the request. Not currently in place. Users should receive the object(s) that they expect to receive. Pending approval for creation of a DP program, the program should be able to document that it delivers what is described. Consultation with program members, creation of use cases, and test access to verify delivery of appropriate digital objects will satisfy this requirement. B6.8 Repository can demonstrate that the process that generates the requested digital object(s) (i.e., DIP) is correct in relation to the request. Not currently in place. If transformations are applied to generate the DIP, these processes must be documented, and the program must be able to demonstrate that the correct transformations are applied. If a program were created, this would be in system design documents. B6.9 Repository demonstrates that all access requests result in a response of acceptance or rejection. Not currently in place. System design documents must specify how a user will be notified whether a request has succeeded or failed. B6.10 Repository enables the dissemination of authentic copies of the original or objects traceable to originals. Not currently in place. System design documents must specify how resources will be authenticated. Program membership, legal mandates, and other user requirements will shape how stringent these processes must be to authenticate documents. C. Technologies, Technical Infrastructure, & Security 47
This section specifies best practices for data management and security and measure the adequacy of the repository’s technical infrastructure. They can be met to some extent by general good computing practices and ISO standard 17799. The Alliance currently has the following technical infrastructure: ● Summit resource sharing system ● Northwest Digital Archives database ● Cloud computing ● Content management system/website ● Administrative databases None of the current infrastructure supports digital preservation since the program is yet to be created. This technical infrastructure has a variety of levels of documentation. Management practices vary somewhat according to program maturity and the degree to which the system is mission‐critical. C1. System infrastructure C1.1 Repository functions on well‐supported operating systems and other core infrastructural software. Not currently in place for digital preservation. If a digital preservation program is approved, the staff and working groups associated with that program will need to perform a thorough assessment of options for well‐supported systems. Software needs to have good documentation, vendor support, and/or strong community support. C1.2 Repository ensures that it has adequate hardware and software support for backup functionality sufficient for the repository’s services and for the data held, e.g., metadata associated with access controls, repository main content. Not currently in place for digital preservation. If a digital preservation program is approved, the program will need to create documentation of the adequacy of its backups that includes what is backed up and how often, logs of backups, disaster recovery plans, testing of backups, and support contracts for backup mechanisms. C1.3 Repository manages the number and location of copies of all digital objects. Not currently in place for digital preservation. 48
If a digital preservation program is approved, the program will need to create a level of expectation for certainty or ambiguity and a regular program to test the number and location of copies of digital objects. C1.4 Repository has mechanisms in place to ensure any/multiple copies of digital objects are synchronized. Not currently in place for digital preservation. If a digital preservation program is approved, the policies and systems must account for synchronization of multiple copies. System design and documentation should specify how long it should take for copies to synchronize and create appropriate workflows. C1.5 Repository has effective mechanisms to detect bit corruption or loss. Not currently in place for digital preservation. If a digital preservation program is approved, systems should detect corruption and loss. An analysis of risks, including hardware failure, human error, and malicious action, should underlie the design for this recovery system. Data losses should be detected and reported promptly. C1.6 Repository reports to its administration all incidents of data corruption or loss, and steps taken to repair/replace corrupt or lost data. Not currently in place for digital preservation. If a digital preservation program is approved, systems should report incidents to administration for action. Incidents, recovery actions, and results should in turn be made available to program members and used to assess the effectiveness of said program. C1.7 Repository has defined processes for storage media and/or hardware change (e.g., refreshing, migration). Not currently in place for digital preservation. If a digital preservation program is approved, it will need to develop a procedure for regularly migrating storage media according to commonly accepted life cycles for major system hardware. In addition, policies will need to take into account the time needed to copy large amounts of data and any impacts that may have on system performance. C1.8 Repository has a documented change management process that identifies changes to critical processes that potentially affect the repository’s ability to comply with its mandatory responsibilities. 49
Not currently in place for digital preservation. If a digital preservation program is approved, the program will need to have a change management process in place so that changes in critical processes are adequately documented. C1.9 Repository has a process for testing the effect of critical changes to the system. Not currently in place for digital preservation. If a digital preservation program is approved, the program will need to have a process for testing its systems and documenting the results. This program will need to be adequate but not over‐elaborate. C1.10 Repository has a process to react to the availability of new software security updates based on a risk‐benefit assessment. Not currently in place for digital preservation. Software security updates can upset system functionality. If approved, a digital preservation program will need to have in place a procedure for evaluating and deciding whether to install security updates. C2. Appropriate technologies C2.1 Repository has hardware technologies appropriate to the services it provides to its designated community(ies) and has procedures in place to receive and monitor notifications, and evaluate when hardware technology changes are needed. Not currently in place for digital preservation. Pending approval for creation of a digital preservation program, the program will need to ensure that its initial hardware technologies, if applicable, are appropriate for the number of members and level of service it will need to provide. A thorough user needs study of the initial and/or potential program members will be key, as will staff with good knowledge of how to specify hardware appropriate to the task. C2.2 Repository has software technologies appropriate to the services it provides to its designated community(ies) and has procedures in place to receive and monitor notifications, and evaluate when software technology changes are needed. Not currently in place for digital preservation. Pending approval for creation of a digital preservation program, the program will need to ensure that its initial software technologies, if applicable, are appropriate for the number of members and level of service it will need to provide. A thorough user needs study of the initial and/or potential program 50
members will be key, as will staff with good knowledge of how to specify software appropriate to the task. C3. Security C3.1 Repository maintains a systematic analysis of such factors as data, systems, personnel, physical plant, and security needs. Not currently in place for digital preservation. Pending approval for creation of a digital preservation program, the program will need to conduct and document this analysis. Some of the risks will be associated with any IT infrastructure; some will be particular to the materials program members submit to the repository. C3.2 Repository has implemented controls to adequately address each of the defined security needs. Not currently in place for digital preservation. Pending approval for creation of a digital preservation program, the program will need to deal with security requirements and be able to document that it has addressed them. This documentation should be among that available to program members. C3.3 Repository staff have delineated roles, responsibilities, and authorizations related to implementing changes within the system. Not currently in place for digital preservation. Pending approval for creation of a DP program, and as part of proposing a full‐blown program, staff roles will need to be clearly defined. Authorizations that relate to system security will flow from these roles and from the risk analysis performed under C3.1. C3.4 Repository has suitable written disaster preparedness and recovery plan(s), including at least one off‐site backup of all preserved information together with an off‐site copy of the recovery plan(s). Not currently in place for digital preservation. Pending approval for creation of a DP program, the program will need to prepare these plans or account for their preparation. As part of this, it will be imperative to involve at least one person with extensive analog preservation experience, including creation of disaster and recovery plans. 51
AppendixF.IRHosting(DigitalAssetManagement)Proposal
Recommendation To create the most flexible solution, the Alliance should host a Fedora Commons instance with one or more interfaces such as: Islandora or Hydra/Blacklight. (Fedora does not have a native access interface.) Streaming support should also be investigated for viability during the pilot phase. The new IT Manager would spend approx. 3 months to set‐up and test system before production with two pilot institutions. Ongoing systems support to be provided with .25 FTE of the IT Manager and .25 FTE of the Digital Services Program Manager for training and support. Pricing Based on fixed expense plus total disk space needed. Heavy streaming might lead to reevaluation of pricing model. ● $1,500 one time setup fee ● $3,000 per year for 50GB or less including backup and bandwidth charges ● $4,000 per year for 100GB or less including backup and bandwidth charges ● $6,000 per year for 250GB, additional 250GB increments available for $1000/yr including backup and bandwidth charges Projected participants We will recruit at least two institutions in first few months during pilot phase with varied use cases. During the Digital Program Working Group’s term (2008‐2009) and the needs assessment process, it was clear that of the forty‐eight institutions who are members of the Alliance or NWDA, a number were looking for systems to host their digital content for public access. These institutions clearly divided into two groups: 1. Institutions that do not currently have a way to host digital content, and are looking to the Alliance for a solution; and 2. Institutions that have local systems for hosting digital content, but are looking for cost savings and efficiencies that they would gain from an Alliance system. (See the DPWG’s final report and recommendations at http://orbiscascade.org/index/cms‐filesystem‐
action/nwda/files/dpwg_report_recommendations_final_rev_20090727.pdf) Based on these results and further conversations with Alliance and NWDA members, we do not anticipate participation problems. Because Fedora Commons is a robust repository system, even institutions that plan to continue to use CONTENTdm for access may choose to participate in our service for a storage solution, since CONTENTdm does not support preservation services beyond OCLC. (E.g. UNC Chapel Hill is currently developing a process to ingest CONTENTdm content into Fedora with iRODS.) 52
We propose a goal to have basic service working in three months with more institutions joining in 2011 Q3‐Q4 and 15 institutions within two years. Technology Amazon Compute Cloud. Basing on the cloud rather than on physical hardware allows purchasing exactly what is needed when it is needed and allows better monitoring, load balancing, and disaster recovery. It also avoids spending time working with depreciation and hardware migrations. Performance will have to be evaluated to determine whether the cloud can be a long‐term solution or if local hardware may be required. Costs Startup (three months): Staff: .50FTE (IT Manager), .25FTE (Digital Services PM) Computing: Varies with installation. During development stage, computer costs will be less than $100 month. As resources are added and use grows prices will rise. Recommend using development stage to determine exact performance options as that will have a major impact on total costs. Ongoing: Staff: .25FTE (IT Manager), .25FTE (Digital Services PM) Computing: Varies with installation. Estimate $550/month including powerful instance on the cloud, backup, storage, and data transfer. Note that almost 1/3 of estimated costs are for provisioned storage so pricing model must take amount of data. Actual starting cost may be lower CPU and disk power can be purchased according to actual needs rather than projecting into the future. Significant streaming use would increase costs. 53
AppendixG.FindingsofDigitalAssetManagementSystems
Evaluation
PowerSolutions
Greenstone and Fedora are both full featured IR platforms that can be used to serve virtually any kind of material. Of these two platforms, Fedora has the better long term prospects. Greenstone suffers from some major shortcomings such as the lack of persistent URLs, access controls, versioning, and submission functions, while Fedora is ultimately more flexible and well suited to the emerging model where specific functionality is added one module or application at a time. However, Fedora has some major drawbacks. By itself, Fedora has no front end for staff or users. While these can be added and packages with some of these functions already built in are available, the work involved in setting up and maintaining this functionality is nontrivial ‐‐ this should take roughly .25 FTE. Vendor options are extremely limited and would not be flexible to meet the needs of a distributed consortia at this time. CONTENTdm was not considered a power solution because of scalability issues encountered by power Alliance users, lack of persistent URLS, no self‐submission, non‐compliant web‐templates, and multiple bugs with their version 5 release (now stable as of July 2010). TraditionalInstitutionalRepositories
Berkeley Electronic Press Digital Commons and DSpace were evaluated in the final round for a traditional institutional repository system. Both systems have basic components necessary to support an institutional repository. The main feature drawback of DSpace is the non‐intuitive submission forms; however, custom submission forms can be created. The DSpace community has also created many open source or vended add‐on such as controlled vocabularies and user contributed tags. Major drawbacks of Digital Commons include: stable links but non‐persistent links, no JPEG 2000 viewer options, and limited to Qualified Dublin Core as a metadata schema. The major advantage of Digital Commons is that they have excellent customer support and are open to investigating new features; however, it is hard to know when those features may be actionable. Another advantage of Digital Commons is its flexibility beyond a traditional IR to include working paper series, image collections, and journal publishing. Also, working with a commercial vendor, they are more flexible to customizations that may be more challenging in a hosted solution within the Alliance. A commercial host for DSpace, however, may have the same flexibility. The most significant drawback in the institutional repository context for Digital Commons was the lack automatically assigned persistent links, which allow for actionable unique identifiers to exist for items regardless of the current system. Persistent links are considered one of the major advantages of an institutional repository system and a large selling point to user communities. Additional services could be purchased to add the functionality to Digital Commons with additional work to map to the metadata for persistent links. However, because it is not native functionality to Digital Commons, we prefer DSpace as a traditional institutional repository system. Many other advantages to Digital Commons could promote it above DSpace depending on identified requirements. 54
Cost:
●
●
●
Digital Commons: based on FTE, starting at $11,000/year for 500 FTE; Price increases are incremental: an FTE of 3000 pays app. $13,000 . Consortial pricing available. DSpace (hosted by WSU) for basic out of the box install: $75 per month / $900 per year for a community/collections space in WSU's DSpace‐based repository and up to 250 GB in storage including setup. DSpace (commercial vendor): Price quoted (2009) $4,000 initial setup plus $18,000 per year maintenance for one terabyte of storage per instance JournalPublishing
Berkeley Electronic Press’ EdiKit and Public Knowledge Project’s Open Journal Systems (OJS) are both excellent platforms for electronic journal publishing. Determination of the best platform across Alliance institutions is not possible because that will depend largely on an individual institution’s resources. Either EdiKit or OJS could support central administration of a publishing platform, with granular controls available to individual institutions for administration of their own journals. Both systems are also logical for use on an individual basis (without relying on centralized service provision from the Alliance). As an open source platform, OJS will be attractive to institutions that have available servers/staff with technical expertise and time to manage the platform. As with any open source product, there’s good flexibility in what can be done to the product, both in terms of branding and functionality. However, for smaller institutions with limited staffing and limited access to technical resources/expertise/time, EdiKit is a logical option for journal publishing. All technical support is provided by Berkeley Electronic Press as part of the license agreement for EdiKit, in a model typical of software‐as‐a‐service.. Though there is limited ability for customizations on the institution side, configurations can be requested (and are promptly made). All fundamental features are consistent with OJS (editorial and workflow management, branding for individual journals, ability to support open access and subscription/PPV model, etc.). For institutions considering purchasing Digital Commons as a repository platform, EdiKit is included (with 5 free journal set‐ups; each journal after is $1,500/year). However, EdiKit may also be licensed separately from Berkeley Electronic Press. Both platforms are recommended, with the usual distinction between open source and proprietary platforms‐ known costs versus unknown costs (which may be time/personnel, not just software/hardware). Multimedia
Evaluating current multimedia systems poses some unique challenges. Colleges and universities are working with multimedia assets in a variety of different ways, and the functional requirements for these different usages can vary widely. Unfortunately, in the current evolving landscape of digital media 55
technology, there appears to be a high degree of inconsistency across current systems in the way multimedia assets are supported and handled. Over the past decade, CONTENTdm has gained a high level of adoption among higher education institutions. Unfortunately, as their customer base has increased, product reliability and development responsiveness have decreased. Because of this, the DST initially chose to exclude CONTENTdm from this category, and identified two newer multimedia products to evaluate, Omeka and Simple DL. However, evaluation work with these two products revealed that while specific features are stronger, both tools are lacking in overall core functionality. As an open source tool, Omeka is vastly more affordable than the other two commercial products. Omeka also offers the best support for user generated content and easy creation of web pages and image “exhibits”. However, Omeka’s reliability is compromised by a lack of core functionality; a majority of features are provided via plug‐ins that don’t appear to be consistently updated with new product releases. Omeka is also heavily oriented toward visual media ‐ it supports the basic ingest and display of PDFs (via plug‐in) but doesn’t offer full text indexing. A new tool on the DAMS landscape, Simple DL shows exciting potential via its superior handling of streaming media and embedded viewers, and overall ease of use. The level of eagerness shown by the developers to‐date is also encouraging. However, vital features still appear to be missing ‐ including image organization tools for users (favorites/galleries, slideshows), batch data editing capability, and support for authentication mechanisms (LDAP/Shibboleth). Also concerning is the fact that Simple DL has yet to develop a customer base or user community. CONTENTdm has some notable weaknesses, but still appears to offer the most extensive core functionality. The product is costly, image/text‐centric (offering poor A/V and streaming media support), heavily reliant on Windows (both server and end user side), and development responsiveness tends to be sluggish. There are also quality and integrity issues, as major product upgrades in recent years appear to have been released prematurely, with insufficient product testing. However, CONTENTdm does have many built‐in features which provide core services to users.. It also offers the benefit of close integration with World Cat, as well as other OCLC products and services. In summary: If alliance members have sufficient staff and resources available and are comfortable with relying on in house technical support and development to maintain and extend a DAM then Omeka is the recommended product of this category. If members are more comfortable with a vended solution the recommended product remains CONTENTdm. ConsortiaProjects
Colorado Alliance ADR (Alliance Digital Repository) The Colorado Alliance expressed interest in Orbis Cascade Alliance participating in their project. However, it was felt better to pursue an independent path for the following reasons: 56
●
●
●
●
●
ADR functionality is based on the Fez front end where our requirements demand a more modern back end with better discovery and streaming capabilities on the front end The technology base we intend to use is inherently more flexible and scalable We had doubts in the readiness to get a program and repository started with them We can focus on the use cases that predominate within the Alliance Developing local expertise will help us with other projects LASR (Liberal Arts Scholarly Repository) LASR uses an aggregation of DSpace repositories using Drupal hosted at http://dspace.lasrworks.org/ with technical support provided by the Longsight Group. (Whitman College is a participant in LASR.) After conversations with Bart Harlow from St. Lawrence, we identified problems in stretching their scope beyond liberal arts institutions and also to wider varieties of content, sustainability, and how the relationship could be mutually beneficial. A decision was made not to pursue collaboration at this time. 57
AppendixH.CrossSearchUtilityProposal
Recommendation The Digital Services Team recommends that the cross‐search utility (previously referred to as the discovery prototype) initiative move forward to pursue an implementation grant from the Institute for Museum and Library Services under the leadership of the Northwest Digital Archives (NWDA) program. This initiative is of great interest to the NWDA membership and has vital importance for the NWDA program's continued sustainability. The NWDA steering team endorses this recommendation and is willing to lead this project and incorporate any viable outputs into its current services and systems. The IMLS National Leadership grant, due February 1, 2011, if awarded, would allow work to begin on the cross‐search utility as early as October 2011 and can be for between $50,000 and $1,000,000 for up to three years. The IMLS application would be in the Demonstration category, which would allow us to build a replicable technical solution and explore how to sustain it without being obligated to do so. This would give the NWDA program and the Alliance an opportunity to solve an important problem without taking on the inherent risk of sustaining a system in the long term. This direction will allow this important initiative to move forward at a pace that will satisfy the needs and expectations of NWDA and Alliance members. Background Importance of the Cross‐Search Utility This initiative, which builds on a prototype developed as part of a 2008 IMLS Collaborative Planning Grant, is of great interest to the NWDA membership and has vital importance for the NWDA program's continued sustainability. In May 2010, all Alliance and NWDA members were asked to look at the visual and textual representations above and indicate their levels of understanding and interest. Of the 26 institutions that responded to the survey, 20 were either interested or very interested in pursuing further work on the utility. Of those, seventeen institutions were current or committed future (FY12) NWDA members. This initiative promises to address the inherently limited audience for encoded finding aids in their present form, a serious limitation for the NWDA and all other EAD programs that is widely recognized within the archives and library professions. Time is of the greatest importance for this initiative: the NWDA program must retain its current members and gain new ones to remain viable, and can only do that by showing the growth and development that its members sought when they voted to become part of the Alliance. The NWDA Steering Team endorses this direction and stands ready to take on this project. A grant‐
writing team has been formed and stands ready to quickly begin the work required to submit a successful application. Uniqueness of the Cross‐Search Utility 58
While this initiative is well in line with needs and concerns beyond the NWDA program, it is a unique effort. A careful survey of other efforts to work with EAD metadata and digital content across the United States shows that this initiative does not duplicate any other multi‐institution efforts. Nor is there any solution offered by a vendor that comes even close to solving the problems that this initiative seeks to solve. Metasearch solutions, for instance, only search; they do not contextualize results and materials in the way that this initiative does. And metasearch of digital content and finding aids, in particular, causes its own problems by putting data and metadata side‐by‐side in search results with no viable way to show how they relate. This is a proven usability problem and reaps almost none of the benefits of structured metadata. Description of the Cross‐Search Utility The cross‐search utility would integrate digitized unique materials with related collection‐level metadata that the NWDA database holds, providing greater descriptive and interpretative context of digitized materials. This fulfills one of the needs identified in NWDA’s Researcher Needs Study and exposes the NWDA finding aids to a wider audience. The focus is contextualization and presentation to enhance discovery, which itself would be optimized for search engines. The cross‐search utility is described in two documents: ● A visual description (See Appendix I.) ● A textual description (See Appendix J.) The cross‐search utility builds on the Discovery Prototype built in 2009 under the IMLS Collaborative Planning Grant (http://nwda.projectblacklight.org/) in that it integrates content from disparate systems and institutions. However, in the concept documents above, contextualization of digitized materials and integration of those materials is much more fully realized than it was in the Discovery Prototype. It is possible that the utility would build on the Discovery Prototype, but the precise technical approach for this initiative has not yet been determined beyond a discussion of its general feasibility. Cross‐Search and Users The Cross‐Search Utility was largely shaped by the outcomes of a Researcher Needs Study, conducted in 2008 as a joint research project of NWDA and Dr. Elizabeth Yakel of the University of Michigan School of Information. The study focused on the research needs of researchers at Alliance and/or NWDA institutions who use digitized archival collections. The study found that these researchers were not well served by any of the sites used in the study, which included a variety of sites that present digitized archival materials for use by researchers. Instead, the researchers wanted a site that combined the access and detail of the digital content sites with the context of the NWDA finding aids site, and that allowed them to conduct very broad searches to find as much material as possible. The cross‐search utility proposes to do exactly that. The report produced as part of the study is available at http://orbiscascade.org/index/cms‐filesystem‐
action/nwda/files/researcher_needs_study_report_final_20090722.pdf. The resulting research paper is out for publication review. 59
Interest in the Cross‐Search Utility All Alliance and NWDA member institutions were surveyed for their levels of understanding of an interest in the Cross‐Search Utility in late May/early June 2010. Out of 48 institutions, 26 responded to the survey. Of the respondents, 22 felt that they understood the concepts as they were presented. However, the comments suggest a bit more murkiness over this presentation and the prototype as presented last summer. Areas of confusion included whether either one included digital content, EADs, or both; whether this tool would also work at the local level; and whether this duplicated another available system or effort. When asked for a level of support, 64% indicated they were interested and 16% very interested, for a total of 80% of respondents. Only 5 were either neutral or disinterested, and no one was very disinterested. In the comments, respondents amplified their interest, but were unclear about why it was needed, how it would work at the institution level, and how it would interact with WCL. Several also indicated that this was a future/theoretical interest. If we look across the respondents, who were they and who expressed the most interest? Very interested (four institutions): Oregon Institute of Technology (NWDA and Alliance) Eastern Washington University (NWDA and Alliance) WSU (NWDA and Alliance) Whitman College (NWDA and Alliance) Interested (16 institutions): Gonzaga University (NWDA) Seattle Municipal Archives (NWDA) Oregon Historical Society (NWDA) Whitworth University (NWDA) Montana Historical Society (NWDA) Alaska State Library (NWDA) Eastern Washington State Historical Society/Northwest Museum of Arts & Culture (NWDA) George Fox University (Alliance, NWDA 2012) Pacific University (Alliance, NWDA 2012) Oregon Health & Science University (NWDA and Alliance) Oregon State University (NWDA and Alliance) Lewis & Clark College (NWDA and Alliance) Western Washington University (NWDA and Alliance) Chemeketa Community College (Alliance) Eastern Oregon University (Alliance) University of Puget Sound (Alliance) 60
Neutral (3 institutions): Evergreen College (Alliance) Saint Martin's University (Alliance) Walla Walla University (Alliance) Disinterested (but very clearly didn't understand the intent; further email interaction indicated some possible support): University of Montana (NWDA) In all the centers of interest were: NWDA (including future NWDA): none very interested, 9 interested, none neutral, one disinterested NWDA and Alliance: 4 very interested, 4 interested, none neutral, none disinterested Alliance only: 3 neutral, 3 interested, none very interested, none disinterested Given these results, we clearly have sufficient understanding and interest to recommend further work. We also clearly have to continue working on communicating our intent for the project and keep our eyes open for similar/duplicative efforts. Selection of Grant Source Thorough research into grant sources shows that the IMLS National Leadership Grant is the source that is most likely to fund this project and allows it to move forward as quickly as is needed. Our IMLS program officer was extremely encouraging based on a review of the visual and textual representations and a detailed conversation. Other possible sources include the National Endowment for the Humanities, the National Historical Publications and Records Commission, or the Oregon LSTA program. However, none of those sources would fund the project before late winter or early spring 2012, which is simply too long a time lag for NWDA and Alliance members. 61
AppendixI.VisualDescriptionofCrossSearchUtility
63
64
65
AppendixJ.TextualDescriptionofCrossSearchUtility
Orbis Cascade Alliance Digital Services Team, Cross‐Search Subgroup Cross‐Search Utility Description 2010 May 26 The Cross‐Search Utility Will: • Search across unique locally held collections in the Northwest in digital asset management systems, institutional repositories, and the metadata in the NWDA database. Nothing else currently does this effectively. • Be a machine interface that exposes materials to search engines rather than a new portal and that assumes entry from a search engine, library catalog or a simple keyword‐only search interface. We will not focus on an elaborate search interface, creating a portal, or explanatory/interpretive materials. • Provide the ability to search for materials across institutions. • Take components of existing systems and present them in a manner consistent with ours and others’ usability testing and user needs studies, particularly: o Search across as many primary resources as possible and allow useful limiting o Combine digital content and all associated metadata in a way that gives description at whatever level of detail is available and context o Give uniform and complete contact information for each holding institution o Give uniform and complete options for getting copies of materials, publication permissions, and more information from an archivist • Solve the problem with finding aids as they are currently presented: valuable and expensive information that is incomprehensible to researchers other than trained historians. The Cross‐Search Utility Will Not: • Perform a meta‐search of secondary or related materials. This is already done in many ways by many other tools. • Provide a way to put digital content and finding aid metadata side‐by‐side with no changes in presentation. This type of presentation is clearly ineffective. If Your Institution Has…. • Any digital objects in a DAM or an IR or finding aids in NWDA? The utility will search and present your materials along with similar and possibly related materials at other institutions, and expose those materials to search engines. • Materials in a DAM, IR, and finding aids in NWDA? The utility will search and present your materials along with similar and possibly related materials at other institutions, and expose those materials to search engines. • Any materials in any of the above systems, it will facilitate basic reference functions for your materials: making contact with your staff, getting copies, getting permissions for publication.