Coffee|Code : Dan Scott - Programming languages

File_MARC 0.6.0 - now offering two tasty flavours of MARC-as-JSON output

dan@coffeecode.net (Dan Scott) — Sat, 14 Aug 2010 15:10:38 -0400

I've just released the PHP PEAR library File_MARC 0.6.0. This release brings two JSON serialization output methods for MARC to the table:

toJSONHash() returns JSON that adheres to Bill Dueber's proposal for the array-oriented MARC-HASH JSON format at New interest in MARC-HASH JSON
toJSON() returns JSON that adheres to Ross Singer's proposal for an object-oriented JSON format (I could only find a sample at this paste - not sure if there's a broader description anywhere, but really -- who needs it?)

The JSON formats should be useful for developers who don't want to have to deal with the overhead and sluggishness of a MARC parsing library (yes, File_MARC, I'm looking at you) just to deal with MARC data. Both formats are round-trippable and compact, which is why I chose to support them.

The use of the json_encode() function bumps the minimum PHP version requirement for File_MARC up to 5.2.x from 5.1.x, which kind of sucks, but given that PHP 5.2.0 was released in 2006, I think it's worth it.

You can install File_MARC using the 'pear' command on most environments as follows:

pear install File_MARC-beta

Moving from Figaro's Password Manager (FPM) to KeePassX

dan@coffeecode.net (Dan Scott) — Fri, 28 May 2010 15:14:07 -0400

I'm one of those people who actually keeps different passwords for every site and service I use. So far I'm up to over 400 passwords, so I'm dependent on a password manager. For a long, long time I have used Figaro's Password Manager (FPM) (and KedPM and most recently FPM2 as continuations of FPM), but now that I have an Android smartphone on which I can browse without wanting to die, I've been itching to get access to my passwords on that. I noticed that KeePassDroid was available, and that KeePassX would work on my desktop. I just had to get from FPM's password export format to one of KeePass's import formats. It turns out that nobody had made that particular leap before (or hadn't shared their conversion script).

Thus... I bring you the FPM to KeePass converter. A straightfoward Python script licensed under the GPL v3 that does a passable job of converting an FPM XML export to a KeePass 1.x or 2.x XML import file. It worked for me, and that's all that I needed; but maybe it will work for you, too.

Seven things

dan@coffeecode.net (Dan Scott) — Mon, 09 Feb 2009 00:56:35 -0500

I was tagged by Lukas for the "7 things" meme, and meant to do something about it, but I've been kind of preoccupied with the new baby and the sprinting toddler and work. Anyway, it seems like a heck of a lot more reasonable than the evil Facebook's "25 things" meme, so I'm going to take a few minutes to try to play along.

I was an early riser until I was around 15 or 16 years old and discovered the surrealists. At that point, I began experimenting with sleep deprivation as a means of stimulating my prose and poetry. This is also when I began drinking coffee. After about a month, I was no longer capable of being an early riser--and the fruit of writing experiments was, uh, not too impressive.
Rather than going directly to university after high school, I elected to take what is now termed a gap year. No trips to Europe for me, though; the goal was to refine my bass-playing and music-reading skills and head to a post-secondary music program. I recorded a few prog-rock tracks in a studio with a fantastic couple of guys (hey Pete and Mike!), but ultimately didn't put enough effort into my bass to carry out the plan. Let me assure you that a year of working night shifts at a convenience store in the entertainment district of a small city is not a waste of time; I can't count the number of experiences that I'm thankful for having had during that time.
Although I roast and grind my own coffee, I'm not a coffee snob. In fact, I possess almost no sense of smell and I suspect that my sense of taste is limited in comparison to most people, and I'm quite happy to drink diner coffee. I cannot stand the taste of Starbucks coffee, however.
The first time I was able to run a full kilometre without walking was when I was eighteen. Since then I've run a couple of 5K races and and a sprint duathlon.
I'm pretty sure I was destined to become a systems librarian. When I was 10, I used to hang out at the local college's computer room until the students would log me onto a completely restricted Gandalf mainframe account so I could pretend to be Matthew Broderick in WarGames. My first real job, when I was 14, was as a "computer page" at the Barrie Public Library Children's Annex. It was my responsibility to oversee the use of the bank of Commodore 64s that the library made available to children, luring them in with games but requiring them to complete their allotment of educational software first. Oh the power.
I occasionally wrote reviews for random CDs that came into the campus newspaper office. Nobody else wanted to review this orange CD called Tragic Kingdom by some West-coast band, so I took it on. I gave it a savage review; I wasn't impressed with faux-ska and couldn't stand the lead singer's voice. Six months later No Doubt's "SpiderWebs" was in high rotation on every radio station in North America (look, folks, that song is repetitive enough without being played twice an hour!). I'm sure that my negative review still gnaws at Gwen Stefani today as she weeps bitterly in her platinum mansion.
In grade one, my report card read Dan is too critical of his classmates. In my defence, if they weren't so stupid--come on, sound it out buddy--I wouldn't have been critical. Okay, not much of a defence.
I am not a very demanding friend. I (almost) never call, (almost) never write, and (almost) never visit. Okay, scratch that: I'm a crappy friend. Most of my close friends found out that we were expecting a second child only through Lynn's Facebook account. I called one couple shortly after Arik was born and his quasi-namesake (one of the Eric's in our life who bring honour to the noble name) asked me after a few minutes: "So, uhh... did we know that you were expecting a baby?". No, no you didn't, and that's not your fault. Man I suck.
I'm really good at arithmetic.

Link your original tagger(s), and list these rules on your blog.

Share seven facts about yourself in the post — some random, some weird.
Tag seven people at the end of your post by leaving their names and the links to their blogs.
Let them know they’ve been tagged by leaving a comment on their blogs and/or Twitter.

Wow, that was fun. Lemme see, I'm going to break the rules and just tag two people: Helmut, because he's one of the only other people who worked on the ibm_db2 PHP driver out of passion rather than as a job assignment. And Gabriel because I like his style.

Oooh... looks like I've got (even more) work cut out for me

dan@coffeecode.net (Dan Scott) — Wed, 16 Jan 2008 22:30:14 -0500

PHP is getting a native doubly-linked list structure. This is fabulous news; when I wrote the File_MARC PEAR package, I ended up having to implement a linked list class in PEAR to support it. File_MARC does its job today (even though I haven't taken it out of alpha yet), but due to its reliance on userspace data structures it's an order of magnitude slower than packages like marc4j so it's not the best choice for processing hundreds of thousands of MARC records... today. It hurts a little that the VuFind project has to use a non-PHP solution for populating its Solr indices - although I'm delighted that they have started using File_MARC for some on-demand processing.

Now, when I get a chance (insert raucous mocking laughter here), I hope to be able to make File_MARC use splDoublyLinkedList and see how it fares with 500K records. Should be good fun! After that, it just needs to be taught how to convert MARC8 to UTF-8, and we'll have ourselves a fully featured standard MARC package for PHP.

FacBackOPAC: making Casey Durfee's code talk to Unicorn

dan@coffeecode.net (Dan Scott) — Thu, 15 Mar 2007 02:06:35 -0400

For the past couple of days, I've been playing with Casey Durfee's code that uses Solr and Django to offer a faceted catalogue. My challenge? Turn a dense set of code focused on Dewey and Horizon ILS into a catalogue that speaks LC and Unicorn. Additionally, I want it to serve as both a proof of several technologies (Solr for faceted searching and Django as a Web application framework) to my colleagues and as a reasonable backup catalogue for when our main catalogue fails (as it all too often does).

I emailed Casey today to tell him that I had a number of patches to contribute as a result of my experiments. It turns out that he's not really interested in pursuing this particular project much further, so he gave me his blessing to take his throwaway code and do whatever I want with it. Thus, the emergence of the FacBackOPAC project on code.google.com. If there's a grant out there for worst project name ever, this project's in the running... Anyways, I have contorted Casey's code so that it supports both Dewey and LC, and with a bit more torture it should be flexible enough to support both Horizon and Unicorn.

Right now I've twisted it all the way to meet my Unicorn needs and consequently have broken Horizon support, but it won't take much to make it support Horizon again - or any other ILS, for that matter. The main requirement is that you have to be able to get your MARC records and holdings out of your ILS. A secondary requirement is to know how to create links to detailed item views in your current catalogue, because this thing does not yet have any current awareness about item status.

There. My itch has been scratched for the time being. Go play with the FacBackOPAC project -- I even have (very) rough documentation on how to get the pieces installed andthe MARC records indexed, although you'll have to dig through the source in the Django catalog tree to overcome some hardcoded strings and URLs for the time being. Don't worry, pulling that hardcoded stuff out of the templates is high on the list of priorities.

So, a huge thank you to Casey for freeing this code and making this possible. For something he considers throwaway code, I've learned a lot from walking through it and making it start to meet my needs. I hope it helps you, too!

Update 2007-03-18:

Edited links to point to the FacBackOPAC project page, rather than the wiki (which is subject to change, and which did -- breaking the dang links in the original version of this story. Argh!)