2009 Fall Preservation and Archiving Special Interest Group

The Preservation and Archiving Special Interest Group (PASIG) comprises educational and commercial institutions that meet semi-annually. Sun Microsystems and Stanford University are the main sponsors of PASIG, so the fall meetings have occurred in San Francisco (with the spring meetings rotating throughout the world). The Sheridan Libraries DataNet award was still recent news during the 2009 Fall meeting, so there was a great deal of interest regarding our initial plans.

I provided a presentation about the Data Conservancy (the name of our DataNet award) that focused on the proposal elements and initial technical architecture. The most important point that I had mentioned was there are significant unknowns and research questions related to data curation infrastructure, particularly related to storage systems. It is tempting to assume that storage is storage and one can simply deposit large amounts of complex scientific data onto storage systems to complete the curation process, but nothing could be further from the truth. Storage is a necessary but far from sufficient condition.

Even at the storage layer, there are fundamental questions regarding failure rates, large-scale system performance, interfaces with preservation and access systems, etc. There are several vendors within the storage hardware and software sector so the ability to span across different vendors is an important consideration as well. More recently, cloud-based storage (and associated services) have become more available as options.

One of the key questions for the Data Conservancy will be comparative advantage. That is, how we do leverage the expertise and capabilities of partners, including commercial storage providers, while focusing on our core areas of expertise such as preservation policies, processes and actions. This type of question could be explored through PASIG.

Both educational and commercial institutions have presented their technology offerings for preservation and archiving through PASIG, which has been helpful in terms of developing a better understanding of the landscape. However, it is my hope that PASIG (and other digital preservation meetings) start to address the "ecological" view of infrastructure development. No single institution will develop the capacity to curate all scientific data, so it's critical for our community to consider the distributed nature of curation, including storage systems. As we build out our local node of infrastructure at Johns Hopkins through the Data Conservancy, we will undoubtedly need to find ways to integrate and connect with other nodes of infrastructure.

Data Conservancy Presentation at Educause

At the 2009 National Educause Conference, I presented a talk about the Data Conservancy, our NSF DataNet funded data curation infrastructure development effort. Rather than write about what I presented, I will point you to the recording for this session at Educause:

The Data Conservancy presentation

10th Sakai Conference and DataNet

I attended the 10th Sakai Conference specifically for Jim Martino's presentation and associated discussions about our NSF DataNet award. It's our intention to build on Sakai's growing ability to connect with distributed repositories to provide access to scientific data within our DataNet framework. In addition to the DataNet specific items, I noted more general trends or observations.

The attendance for this Sakai conference was at least as much as prior conferences (so much so that they ran out of t-shirts), which is especially impressive given the current economic climate and associated travel restrictions. The number of members and institutions using Sakai continues to increase. Based on conversations with a couple of Sakai Board members, I also felt that Sakai 3.0 was moving in interesting ways that emphasize learning in different ways. The fact that "management" is part of the term "course management system" or "learning management system" has always struck me as odd. Do courses or learning really need to be managed? Within the Sakai community, there appears to be a greater awareness, even embracing, of the collaborative aspects of learning for which Sakai might have been conceived of in the first place.

Open Repositories 2009

In mid-May, I attended the annual Open Repositories meeting for the third time. I noted after attending this conference for the first time in 2007 that it was the most practical conference that I had attended in some time. And the last two years have done nothing to disabuse me of that notion. In fact, if there were anything that I would complain about, it would be that the conference is a bit overwhelming because it is so information rich.

I should note that Sayeed Choudhury and Elliot Metsger also attended this meeting and have already blogged about it.


Data Curation, Archiving, and Preservation

Because of the Data Conservancy (our DataNet project currently in the start-up phase) and our DataPub project currently underway, curation of and long-term access to data is of key importance to the Digital Research & Curation Center (DRCC) and the Sheridan Libraries in general. Many of the presentations covered issues of interest in this area. I'll highlight a few of them below.

As Sayeed mentioned in his post, Michael Witt of Purdue spoke about research into the development of data curation profiles. This work is a collaboration between Purdue and UIUC's Graduate School of Library and Information Science. Their approach is based on discussions with researchers and employs an initial unstructured interview to get the conversion started. One of the most interesting findings thus far relates to issues of the data sharing (with whom, after what activities. Michael presented an earlier version of this work at a Sun Preservation and Archiving Special Interest Group (PASIG) meeting. More information can be found on the project site.

John Kunze of the California Digital Library and our own Sayeed Choudhury both spoke in a session devoted to the recommended NSF DataNet projects. John spoke about the Data Observation Network for Earth (DataONE) project, led by University of New Mexico. Sayeed spoke about our project, the Data Conservancy. My focus was on the IT and data frameworks of the two projects. The approaches are different in many ways and it will be interesting to work together to establish the kinds of data management partnerships envisioned by NSF in the creation of the DataNet program.

In addition to the talks, Sayeed and I pulled together a birds of a feather session, which he was unfortunately unable to attend. I was there to represent the Data Conservancy's process and approach. John Kunze and Stephen Abrams, both of whom I was fortunate enough to wrangle at the last minute, represented the perspective and approach of DataONE.

Simple Web Service Offering Repository Deposit (SWORD) and the Open Archives Initiative Object Reuse and Exchange (OAI-ORE, or ORE for short) are two relatively recent developments meant to, respectively, reduce the burden of content deposit and improve the description and exchange of resource aggregations (think compound/complex objects) on the Web. We are employing both of these technologies in our DataPub (curating published data) project. Elliot has done a nice job of highlighting some of the ORE presentations in his post, so I will just add a few comments about the SWORD talks.

Pablo Fernicola gave a presentation describing work on an authoring add-in for Microsoft Word on the Windows platform. The add-in, currently in beta, will support ORE, SWORD, and the Publishing tagset of the NLM DTD. We have been working with Pablo on the ORE components of the add-in. This technology will allow an author to create a document, link it with data and rich media, describe the relationships of this components, and submit the package to a repository -- all without leaving Microsoft Word. While other approaches will be needed for other authoring environments (e.g., LaTeX), these tools go a long way to lowering the barriers to contributing and reusing content.

Adrian Stevenson and Julie Allinson shared a talk describing ongoing work in the second phase of development (SWORD2) and some of the history behind the development of the original SWORD protocol specification and implementations. It is now possible to deposit content into a properly configured Fedora, DSpace, or Eprints repository through Facebook, a web client, and a desktop client (among others). As I mentioned previously, the Microsoft Word will soon support SWORD deposit via an add-in.


Repository Challenge

The Repository Challenge started last year at the Southampton Open Repositories meeting and was organized by David Flanders, then of the JISC-funded Common Repositories Interface Group, with the goal of getting "developers working in small teams to try to quickly pull together established platforms and services to demonstrate how to achieve real-life, user-relevant scenarios and services."

This year's Challenge was again organized by Flanders, now of JISC proper.

The Repository Challenge winner this year was Tim Donohue of UIUC. Tim used JavaScript (JS) to implement a system he called "Mention It". This JS library allows a web page designer to embed into a web page an aggregation of mentions of a specified string on Twitter, FriendFeed, Technorati, and Google Blog Search. Among many other uses, this would allow repository developers to embed the display of mentions for a digital object by specifying its splash page, item, or Handle URI.

The runner-up, Rebecca Sutton Koesar of Emory, created FedoraFS, which combined a Fedora Commons repository with FUSE (Filesystem in User Space) to support access to repository content as if it were in regular files. For example, a PDF file stored as a datastream within a Fedora digital object could be accessed with a standard desktop PDF viewer. Her entry video is available on vimeo.