Coffee|Code : Dan Scott - Coding

Why I donated to the Software Freedom Conservancy

dan@coffeecode.net (Dan Scott) — Mon, 26 Dec 2011 09:15:23 -0500

A few days ago I made a small donation to the Software Freedom Conservancy, a 501(c)(3) non-profit organization registered in the United States. There are many organizations to which I could have donated, and indeed Lynn and I have donated to a number of charities again this year, but I felt it was important to direct some funds to the Conservancy for a number of reasons - which I will attempt to describe and hopefully convince you as well.

First, for those who know that the Evergreen open source integrated library system is a member project of the Conservancy and the the project on which I invest much of my professional and person time, an obvious question might be: "Why didn't you just donate to Evergreen?". Donating to Evergreen does result in a small percentage of those funds being directed to the Conservancy. Currently, Evergreen directs 5% of its income to the Conservancy, but I feel that even with $20,000 passing through the project's hands for the purposes of the 2012 Evergreen conference, that $1,000 that goes to the Conservancy is far below the value our project has received in return in the form of Conservancy services.

One of those services is the provision of a trusted third-party home for project assets such as the aforementioned finances, but also including domain names, trademarks, logos, and (if desired) copyright. While distributed ownership of these assets is not a problem for projects when everything is going fine, personal disputes, a change of business strategy, or new ownership of a contributing company can lead to severe difficulties for a project. Evergreen's sister project, Koha, found itself forced to change its domain name and fight trademark battles over its very name when one company adopted an aggressive business strategy.

Another service from which Evergreen has thus far derived great benefit is access to legal counsel familiar with software freedom issues. In September the Conservancy added Tony Sebro as General Counsel to offer basic legal assistance to its member projects. The Conservancy was most recently involved in a discussion about Evergreen documentation licensing that evolved from an unfortunately adversarial position to, shortly after the Conservancy became involved, a mutually satisfactory agreement. I believe this result was due not only to Conservancy's legal expertise and familiarity with the specific licenses in question and the general mechanism of granting licenses, but also with their ability to understand the goals of the project and its participants in helping to guide all parties to their desired goals.

The Conservancy also has a wealth of experience to draw upon to offer guidance expertise on many matters that free software projects have in common, but which each project tends to rediscover on its own. For example, the Evergreen project has been able to run conferences on an annual basis for the past three years, but has historically relied on Equinox's willingess to assume the financial risks when signing venue contracts. This year, due to the positive results of the previous conferences, the Conservancy was able to provide the deposit for the Evergreen 2012 conference in Indiana. While personally I deeply appreciate the role that Equinox has played in helping to build such a core part of our community experience, it is an important step for our project that the Conservancy be able to assume this role.

In addition, the Conservancy's experience with various conference management packages and the payment fees associated with online financial services such as Google Checkout and PayPal provided some important guidance early on in the Evergreen conference 2012 planning process. That advice probably paid for itself!

I expect that the Evergreen project will continue to benefit from our membership in the Software Freedom Conservancy as we work towards a mechanism for electing members of the Evergreen Oversight Board and continue growing and evolving the project. The $1,000 or so that the Conservancy has earned as a result of the 5% of revenue that Evergreen directs its way is far below the value that we have derived from our relationship thus far, and that is why I have chosen to donate to the Conservancy again this year.

P.S. As a 501(c)(3) non-profit, donations to the Conservancy are tax-deductible for American citizens. As a Canadian, this particular benefit does not apply to me - however, the rest of the benefits that the Conservancy provides to free software projects are international in scope and deserve to be supported.

What does a system librarian do?

dan@coffeecode.net (Dan Scott) — Tue, 22 Nov 2011 16:50:23 -0500

Preface: I'm talking to my daughter's kindergarten class tomorrow about my job. Exciting! So I prepped a little bit; it will probably go entirely different, but here's how it's going to go in my mind...

My name is Dan Scott. I’m Amber’s dad. I’m a systems librarian at Laurentian University.

Today you’re going to learn what a systems librarian does. Exciting, eh?

I bet you have all been to a library. When you think about a library, what do you think of first?

Books

Yep - books! I do a lot of work with books! Can you guess what sorts of things I do with books?

Choose which books to add to the library: Selection
And when we get new books, where do we put them? : Organization
And when we have too many books, choose which books to give away: Weeding

But there’s more to libraries than books! What else can you think of that are in libraries?

Movies
Music
Magazines
Newspapers
Puppets
- Yes, puppets
Computers

Ah, computers. That’s where I really spend a lot of time. When I was a little boy, I was a voracious reader - I would read anything, including cereal boxes and encyclopedia - and I was fascinated by computers. Completely obsessed. So, naturally, when I went to high school I also got a job at the children’s library as a "computer page". I was a big kid, and I helped all the little kids use the computers at the library.

Now that I’m grown up, I’m still doing pretty much the same thing - except now I’m helping the adults use computers. Except now I’m helping them by making it easier for them to get the books or magazines or music or movies or puppets (yes! puppets!) that they need; and a lot of the time, they don’t even have to come to the library. They can read or watch or listen to whatever they need right on their computers - and sometimes they need help, but that’s what I’m there for.

Current state of academic reserves support for Evergreen

dan@coffeecode.net (Dan Scott) — Wed, 07 Sep 2011 23:09:24 -0400

One of the relatively frequent questions that I run into with Evergreen is "Does Evergreen have an academic reserves module?" And the answer is: well, yes, and no. There is no official academic reserves module that is part of the standard Evergreen package that you download and install from http://evergreen-ils.org.

However, I am aware of two free-and-open-source modules that are available as extensions to Evergreen:

A relatively simple, straightforward module, written by my colleague Kevin Beswick, is in use at Laurentian University and recently was adopted by the emily carr university of art + design. It builds on Evergreen's bookbags feature to organize reserves of physical items by class code and instructor name. The module for that code--a mix of PHP, Dojo, and SQLite--is available on Github, and you can see it in action at Laurentian University
Syrup is a more sophisticated reserve system (you know it's a serious project when it has a name!), which supports all kinds of features - such as mixes of electronic and physical materials, organizing course content by arbitrary groupings (e.g. readings per week), limiting user access to the content of specific courses based on LDAP integration, and much much more. You can see a running instance at the University of Windsor and the code (primarily written in Python) is freely available from the Syrup git repository on Evergreen's git server. If you need help getting up and running, Syrup's mailing list is probably a good place to start.

So, there are at least two choices for academic reserves for Evergreen. Go ahead and pick the one that meets your needs!

The wonderful new OpenLibrary Read API and Evergreen integration

dan@coffeecode.net (Dan Scott) — Thu, 02 Jun 2011 16:06:24 -0400

Back in early May, I was in San Francisco for Google I/O. I had booked an extra day with the hopes of either doing some site-seeing or meeting up with the OpenLibrary team. After firing off an email to find out if anyone there was interested on working on some tighter integration between OpenLibrary and Evergreen, the answer from George Oates was an enthusiastic "Yes!". So, we spent a beautiful sunny day inside the Internet Archive headquarters discussing possible directions for this integration. Alcatraz, you can wait for my next trip...

As it turned out, the timing was great. I had spent a day hacking on the OpenLibrary "added content" module for Evergreen during the Evergreen hackfest (which I spent in an airport due to an eight-hour fog delay... different story), so I was quite familiar with the existing OpenLibrary Book API and their patterns of use were fresh in my brain. The biggest problem with the existing Book API, from my perspective, was that I had to make two calls for each work that I was interested in retrieving information about; one call returned the data (stable elements) and one call returned the details (unstable, but quite interesting elements like the table of contents, excerpts, etc).

The OpenLibrary team had this in their sights as well - but they wanted to tackle a bigger target. Rather than making one or more calls per work, they wanted to expose an API that would let users request info for multiple works in one shot: the Shotgun API (known amongst more polite company as the Read API). Loosely modelled on the Hathitrust API, it would also focus on exposing URLs for reading or borrowing (using the relatively recent OpenLibrary borrowing program) exact matches or similar editions. It sounded great, and we spent the afternoon fleshing out how we wanted it to look and work. My role was largely that of the third-party developer - the API customer - and we had great discussions.

Working code wins

Of course, discussions are one thing, and working code is another. OpenLibrary developer Mike McCabe was riding shotgun on the development of the Read API, and once he had enough working code in place, he contacted me to ask me to start developing against it. It was the usual development process: I started with a hard-coded sample JSON output, then as Mike pushed more functionality into a server environment I was able to test and expand my client-side code.

So where are we now? I can vouch that working with the all-in-one Read API, as a developer, is sweet. All of the data elements are readily visible in sweet, sweet JSON, in a single call, and it is utterly simple to pull the bits that you want to expose. I had been trying to pull together ebook links and the like from the Books API, and the use of the items list makes that absolutely painless for developers. Kudos!

Evergreen has a largely rewritten OpenLibrary added content module built against the Read API sitting in the Evergreen working repository user/dbs/openlibrary-read-api branch. As the Borrow and Read functions depend on IP address range matching, I have added the ability to proxy the Read API requests via the Evergreen server - so that if an Evergreen institution has special access rights to the OpenLibrary collection, their patrons will see the appropriate levels of access in the catalogue. Oh yes, the catalogue; as we were already using OpenLibrary by default for cover art, tables of content, and excerpts in Evergreen since the 2.0 release, the major difference that will be visible to Evergreen users will be in search results:

As you can see, if you have left the OpenLibraryLinks variable turned on in the result_common.js file, Evergreen will search for a matching record in OpenLibrary and tell you if an online version is available. It tells you whether the online version is an exact match, or similar, and will also expose items that you can borrow from OpenLibrary. Given the preponderance of print materials that still remains in our collections, and our users' general preference for anything electronic, I think this will be an extremely popular feature.

Moving forward

There are a number of areas that could use more polish and tender loving care. First and foremost, OpenLibrary supports matching based on ISBNs, LCCNs, OCLC numbers, and OpenLibrary IDs; right now, the Evergreen support is based strictly on ISBNs, which of course don't exist for many of the older materials in our collections. So a fruitful direction would be to take the regular dump of data that OpenLibrary thoughtfully provides (yay for open data) and use that to augment our records to include OpenLibrary ID numbers to use as a match point.

There is the small matter of merging these changes back into Evergreen proper. I developed against the Evergreen 2.0 branch because I wanted to be able to put this code into production as soon as possible, so there will be a tiny bit of merging pain to get this into master and backported properly. However, the changes are quite localized and should be agreeable, so hopefully this will not sit in a branch for too long.

At this early stage in the Read API's release, I have also found that it can be a bit slow to respond to requests containing a number of identifiers (or perhaps a large number of records and items). It is to be expected that functionality comes first and optimization comes later, so I have great hopes for improved performance once the Read API settles down.

Of course, once you have the Read API, you need an Write API - and I hope to be able to help pilot that as well, because the potential communal benefit of a Write API for library systems that have integrated with OpenLibrary is huge. Imagine a system where, when you ask for added content based on a given identifier, if the system says "Huh, I don't know anything about that identifier" it follows up with "Hey, can you POST what you know about it to this URL?". OpenLibrary could then run its algorithms and either add an edition to an existing work or generate a new work. We should also be able to expose OpenLibrary's metadata editing tools for our users, so they can flag bad cover art, or add a table of contents to works that they are passionate about, or post a favourite excerpt... Enabling a bi-directional give and take between systems has the potential to quickly make OpenLibrary a huge knowledgebase of open data. It would be a great boon for libraries, and I hope we can make it happen.

Update 2011-06-02 21:54 EDT: The omission of Mike McCabe's name has been corrected. Also, I forgot to thank my employer, Laurentian University, and the University of Windsor for allowing me to invest some of my time on strengthening Evergreen's ties to OpenLibrary. I believe this is the beginning of a solid, mutually beneficial partnership!

Farewell, old Google Books APIs

dan@coffeecode.net (Dan Scott) — Thu, 26 May 2011 22:08:54 -0400

Since the announcement of the new v1 Google Books API, I've been doing a bit of work with it in Python (following up on my part of the conversation). Today, Google announced that many of their older APIs were now officially deprecated. Included in that list are the Google Books Data API and the Google Books JavaScript API. These APIs will be retired as of December 1, 2011. (Thanks to jgeerdes in the #googleapis IRC channel for the heads up today).

There already has been some outrage expressed over the switch to new APIs; five months is not a lot of time to shift gears if you've built a significant architecture on top of the old APIs. But I have some sympathy for Google, in this case; the new "Discovery" APIs are based on a common, consistent architecture that will be easier for them to document, maintain, manage, and ... monetize, of course. (Good time for full disclosure, I suppose: I am a Google stockholder.)

So far, the only major concern I have with the new v1 Google Books API is one missing function that was available in the Data API: the ability to do a full-text search of a custom bookshelf. Accordingly, I've filed a bug in the AJAX APIs issue tracker. Here's hoping that the deprecation of the old APIs enables Google to focus on their anointed APIs on all fronts: documentation, features, and support. Bug 587 should be a good testcase.